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I. INTRODUCTION 

This report describes and documents the research conducted at the 
Naval Postgraduate School (NPS) during FY 76-77 on the System Test Methodology 
project sponsored by the Naval Air Development Center (NADC) . The project 
began in FY 75-76. Previous reports were "System Test Methodology/' Vol. I, 
NPS55 Ss 75072A and Vol. II, NPS55Ss75072B, July 1975, published by the Naval 
Postgraduate School [1] . Several publications in the open literature have 
resulted from this research [2, 3, 4]. This work covered three areas; 

a. Software error simulation and analytic models. 

b. System test simulation model. 

c. Partitioned tests for software error analysis. 

Computer programs were developed and provided to NADC for the simulation and 
analytic models of a. Of the three areas, it was felt a. had the greatest 
potential for improving software reliability. Consequently, the following 
efforts were undertaken in FY 76-77 : 

• Improvement of the software error simulation model. 

• Validation of the software error simulation model. 

• Analysis of program complexity measures. 

These topics are covered in Sections II, III, and IV, respectively. A brief 
overview of each topic, in the order listed above, will be given here. 

® Sections of the simulation program which involved execution and 

repair times were removed because: (1) current research interests 

are structural characteristics and error detection properties of 
programs, rather than timing aspects and (2) CPU time required by 
these parts of the program were too high in relation to the need for 
this information. The new error insertion feature (simulates the 
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possibility of new error insertion as a result of error correction) 
was removed because this process is not well understood. Therefore, 
it was difficult to prescribe the appropriate probability distribu- 
tions and conditions which should govern error insertion in a 
simulation. User instructions are provided in Section II. A program 
listing of the simulation model is included with this report. 

o Validation tests of the simulation model were performed with respect 

to: (1) mean number of errors seeded, (2) probability of arc traversa 

and (3) number of arc and path traversals. Validation tests were 
conducted with respect to the analytical model . We did not conduct 
validation tests of either the analytical or simulation model with 
respect to the error detection process in real programs, although 
Naval Tactical Data System program structures and error parameters 
were used in some of our studies. Thus, validation tests were limited 
to a test of the correctness of our concept of error detection. The 
analytical model was used as the standard for comparing simulation 
results. Independent appraisals by NADC and NPS personnel have con- 
cluded that the analytical model's equations accurately represent the 
error detection process as originally conceived. Validation with 
respect to actual program error detection processes will be attempted 
in future research. 

o The third area involved using the models to study the relationship 
between program structure and error detection properties. Naval 
Tactical Data System programs were used for this investigation. 
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II. DESCRIPTION OF ERROR SIMULATION PROGRAM 



Summary 

This program is designed to simulate the error detection process 
in a portion of computer software, represented here as a directed 
graph. The graph consists of nodes representing merging or branching 
points in the software and arcs representing the sequences of instructions 
between the nodes. There are no real limitations on the topology of 
the graphs which can be handled by the program except that multiple arcs 
between the same two nodes are not permitted. If such arcs are desired, 
they can be artifically handled by introducing additional dummy nodes 
on each of the multiple arcs. The program is currently dimensioned 
for thirty nodes. This could easily be changed although some formats 
would also have to be modified. 

In the specif ication of the graph, the user can input the length 
(number of instructions) of each arc or allow them all to be set 
internally to the same length of 10. 

The locations of the errors in the graph can also be specified by 
the user, or the program can place the errors by a random process. 

When this is done random numbers representing the distance between 

errors are drawn from the exponential distribution. The errors are 

then placed in the arcs corresponding to these randomly selected 

instructions . 

The MAIN program simulates the process of running an input through 
the directed graph. Each input finds all errors on its path in a 
single pass. The single input is terminated when it reaches a node 
from which no arcs emanate. 
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The selection of which arc to follow from a given node is made 



randomly (uniform) using IZ as the random number seed. The seed is 
used to generate a new random number which serves as the next seed. 

The seed IZ is not reset to its initial value when the graph is re- 
seeded with errors. Thus the random process IZ continues to run 
sequentially until the run ends. 

For the graph which is input, the data specifies (among other items) 

a) the number of inputs per replication 

b) the number of replications per seeding 

c) the number of seedings 

The main elements of the model are discussed in more detail in the 
following sections. 

1. error seeding 

2. branching 

3. error finding 

4. data input 

5. computations performed 
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1. Error Seeding 



The user has the option of planting errors in the arcs of the directed 
graph or he can allow the program to seed the errors randomly. If the user 
chooses to let the program place the errors, he must specify the parameter 
MEANER as part of the input data. This parameter is used by the program as 
the mean of the distribution which determines the distance between success- 
ive errors. It is conveniently interpreted as the mean number of instructions 
between errors . 

Since the arc lengths and distances between errors are treated 
internally as floating point numbers, it is not necessary that arc 
lengths be integer, although that is the proper interpretation when the 
length is measured by the number of instructions. It may be desirable 
to measure length in some other way related to the complexity of the 
instructions and the program permits this. 

Unless the user specifies the individual arc lengths X(K,J), they 
will all be set internally to 10 (line 95) . The random selection of 
arc lengths is not a feature of this program but a simple modification 
would permit this, if desired. If so, a random number seed (say lY) 
would have to be specified above line 95. To avoid correlation with the 
other random processes in the program the symbols IX, IW, and IZ should 
not be used as the seed. 

The random seeding of errors is done in the subroutine SEED. 

The arcs of the graph is assumed to be arranged in "natural order" 
so that arc (i,j) precedes (k,l) if i < k, and arc (i,j) precedes 
(i,l) if j < 1. Each arc (i,j) has a specified length X(I,J) and 
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it is convenient to think of these lengths as being laid out sequentially 



on the real line beginning at the origin. 

Using the seed IX, a random number ERl is drawn from the exponential 
distribution with mean 1 and rescaled by multiplying it (in floating 
point) by the parameter MEANER to yield XERl. Thus the quantity XERl 
is a sample from the exponential distribution whose mean is MEANER. 

The quantity XERl is used as the distance from the last seeded error 
to the next. A comparison is made to ascertain in which arc the error 
should be placed and the process is repeated with a new value of 
XERl until the location of the "next" error falls beyond the total 
length of all arcs combined. At this point the seeding process is 
complete and the SEED subroutine returns control to the main program. 

The variables set in SEED and placed in common with the MAIN 

program include 

/ 

ISEED(I,J) = The number of errors seeded in the arc ij for 
every arc 

SVSEED(I,J) = A saved copy of ISEED(I,J) 

NINST = total number of instructions 

NSEED = total number of errors seeded. 

When errors are planted by the user, the subroutine SEED is not called, 
and the variables NINST and NSEED are not computed. 



II-4 



2. Branching 



This section discusses the branching mechanism which governs the 
path taken by an individual input traversing the directed graph. 

Node one is assumed to be the input node to the directed graph and 
any node having no arcs leaving it is a terminal node. To preclude 
endless cycling, it is necessary that the graph have at least one 
terminal node which can be reached from node 1. Except for node 1 
no special ordering of nodes is required. An arc can go from any 
node to any node. There is no special significance to an arc which 
begins and ends on the same node. Such self-loops may be used to model 
repetitive processes which are repeated a variable number of times. 

The momber of repetitions is not controlled by the user but is 
determined by the branching process which randomly selects the next 
node in the sequence of nodes encountered by a particular input. 

/ 

The branching process from a particular node is begun by 
counting the number of arcs NUMSUC which emanate from the- current 
node designated as NODE. Then using the seed IZ, a random number U 
is selected from the uniform distribution on the interval 0 to 1. 

The quantity K = 1 + NUMSUC*U is computed. Note that K can assume 
any integer value from 1 to NUMSUC and that these are all equally 
likely. 

It is convenient to think of the arcs emanating from NODE as being 

arranged in natural order as the first, second, third, ..., NUMSUC. 
t h 

Then the K — arc emanating from NODE is selected as the node to which 
the input advances. 
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3- Error Finding 



The number of errors placed in an arc ij is ISEED(I,J). When the 
traversal of a particular arc is simulated by the program, it is 
assumed that all errors in that arc are detected by that input and 
corrected before subsequent inputs are run. Thus after traversal 
of arc ij the variable ISEED(I,J) is set to 0. The variable SVSEED(I,J) 
is available for replacing the original errors when desired. 

The total number of errors detected by any input is cumulated and 
recorded as NFIND (line 166) - 

Other error finding mechanisms are possible and may prove realistic 
if sufficient data becomes available to reveal more about the error 
finding process. For example, it may be more realistic to assume 
that when an arc is traversed its errors are exposed to detection but 
may or may not actually be found depending on some random process. 

An assumption of this type could be accommodated in this model if 
some modifications are made. 



II--6 



4. Data Input 



The required data is described below. Each card or group of cards 
is described in a separate section. 

(a) MINPUT, NUMOUT, NREPET, MEANER, N. The format is 

(515) 

MINPUT is the number of different inputs desired within each 
replication (dimensioned 50) 

NUMOUT is the number of replications within each repetition 
NREPET is the number of repetitions or re-seedings. With each 
reseeding NUMOUT replications are performed and in each of those 
there are MINPUT inputs . 

MEANER is the mean distance between seeded errors 

N is the number of nodes in the graph (currently dimensioned for 

30) 

(b) The graph structure is read in as described below. The number 
of cards is variable but can not exceed N + 1. 

Each card has format of (1615) although typically many fields will 
not be used. 

The first field identifies a node from which arcs emanate. 

The second field gives the number of arcs (^ 14) . 

The remaining 14 fields (if required) identify the nodes to which 
these arcs go. 

The above information is repeated for each node from which arcs 
emanate . 

This section of data is terminated by a card with 99 punched in 
columns 4 and 5. 
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(c) The following cards are optional . If used, they specify the 



arc lengths. 

The format is 215, 7(15, F5.0). 

The first field identifies the from node. 

The second field specifies the nimber of arcs. 

The next fields are used in pairs and have the following 
meanings. 

first field, identifies a node. 

y 

second field, specifies the arc length. 

This section is terminated with a 99 in cols 4 and 5. This card 
is not optional . 

(d) If the user wishes to plant the errors in arcs of his choice 
instead of letting the program seed the errors randomly, the following 
cards are used. 

The format is (1615) . 

Field one specifies a from node 

Field two specifies the: number of arcs emanating from the node 
in which errors are to be planted. 

The next fields are used in pairs. 

“ first in pair, identifies the to node 

- second in pair, specifies the number of errors in the arc 
just defined 

This section is terminated with 99 in cols 4 and 5. (not optional) 

(e) The last card is used to specify an output option. The 
variable name is MOOT. 

The format is (15) 
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If a 0 is punched, only summary output is given. 



If a 1 is punched, detailed information about the seeding, the 
paths, etc is given. This should be used only for small values of the 
product NUMOUT * NREPET. 

If this product exceeds 25, the program sets MOUT=0 as protection 
against extensive output. 
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5, Computations Performed by the Program 
a. As discussed in section A-1 the program simulates MINPUT inputs 
for each of NUMOUT replications and the entire process is repeated for 
NREPET seedings. This section describes the computation performed in 
producing the summary output. 

Let i = 1 , . . w MINPUT 
j = 1 , - . . , NUMOUT 
k = 1, . . . , NREPET. 



X. . = number of errors found on input i, replication j of seeding k. 

13 K 

For each seeding k a summary is produced giving the average number 
of errors found for each input and the standard deviation of the number 
of errors found on each input. These are defined as follows: 



INPUT NUMBER i 

AVE NUMBER ERRORS FOUND 



X. , 
i*k 



NUMOUT 

I 

j=i 



X / 

ijk NUMOUT 



STD DEV 



i ‘k 



-,1/2 



NUMOUT 



L j-1 



I (X. - X ) / 

13 k i*k 



(NUMOUT- 1) 



After all NREPET seedings have been analyzed a summary output is 
produced giving the average number of errors found for each input and 
the standard deviation of the number of errors found. These quantities 
are defined as: 



SUMMARY FOR INPUT i 



AVERAGE ERRORS FOUND 



X . 

1 ‘ 



= I I X..V 

. f" 13 k 
: k 



(NUMOUT) • (NREPET) 
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STANDARD DEVIATION 



a . 

1* 



1 1 (x,. -x. )^/ 

Y , Ilk 1 • • 



^ 1/2 



j 



((NUMOUT) • (NREPET)-l) 



b. To assist in relating these computations to the the program, the 
following section is included, 
for each input i 

IFOUND( ) = J X. for k given 

.Ilk 
3 

V 2 

SFOUND( ) = ) (x. . ) for k given 
.Ilk 

CUMSQR( ) = J — cumulative number of (squared) 

, ii k 

J k 

errors found for all replications 
and seedings. 

SVAVE( ) = y Jx.., 

^ ^ i:)k/NUMOUT 



T AVE — y y X 

^ ^ ijk/(NUMOUT). (NREPET) . 

J k 

The quantity hhe standard deviation is computed using the 



following relationships: 



n 1/2 



I y(x.., - X. ) "^/((MUMOUT) • (NREPET) -1) 

‘r h Ilk • 



j k 



1/2 



2 ^ 

[y yx.. - (NUMOUT) • (NREPET) xT ]/( NUMOUT) • (NREPET) - 1) 

A t 1 " 



j k 



[CUMSQR( ) - (NUMOUT) (NREPET)(TAVE) ■■]/( (NUMOUTXNREPET) -1) 



1/2 
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III. VALIDATION TESTS 



This section describes validation tests of the simulation model; the ana- 
lytical model [1] was used as the standard. The test results are preceded 
by a brief description of the analytical model. Hypothesis tests were con- 
ducted of: (1) mean number of errors seeded, (2) probability of arc traversal 

and (3) numbers of arc and path traversals. Tests of mean number of detected 
errors were not performed because an efficient method of computing the variance 
from the analytical model was not available. The standard deviation of detected 
errors was needed to calculate confidence limits. Although the simulation model 
computes an estimate of the variance, this calculation could not be used because, 
without the analytical model variance, it could not be validated. However, a 
notable achievement has been the development of an algorithm for computing the 
variance of number of errors detected from the analytical model [5] . Work is 
proceeding on obtaining a solution in closed form. When a computer program is 
available for the closed form solution, validation tests will be conducted of 
simulation model mean number of detected errors and the variance. 



Analytical Model 

The analytical model [1] computes the exptected number of detected 
errors for each input. Designating the original expected number of errors 
in arc ij as y. ., for the arc between nodes i and j and p. . as the 
probability of traversing arc ij one or more times, the expected number 

The expected 



of detected errors in arc ij for the first input is y. .p. . 

ID 13 



number of errors remaining in arc 



ij after the first input is y . . (1-p. .) . 

ID ID 



The expected number of errors detected on the second input is y . . (1-p. .)p. .. 

th ID ID 

The expected number of errors detected on the k — input is 



e (k) = y . . (1-p. . ) ^ ^p. . 

ID ^iD ID 



( 1 ) 



When the expected number of detected errors is added over all arcs, we have 

k-1 



E(k) = I U (l-p )• 
ij 






( 2 ) 



th 



for the total expected number of detected errors on the k — input. The 
initial y^^ can be interpreted as the mean number of errors per arc which 
is originally present, in which case the equal for all arcs, 

or as a specified number of errors in each arc. In the latter case, the y^^ 
will be different. When comparing simulation and analytical results, y^^ 
is used in (1) if the simulation is repeated for a number of seedings and 
y^j is used in (1) if only one seeding is used. 
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When an actual program is available for analysis, the number of source 

statements in an arc s . . and the mean number of source statements between 

ID 

errors M (obtained from historical module error records) can be used to 

obtain the p. . in (2) by the computation p. . = s. ,/M. Then (2) becomes 
ID ID ID 

E(k) = ( I (3) 

ij J J J 

If it is desired to calculate (3) as the fraction of original errors detected, 
then (3) is divided by U, the expected number of original errors in a program. 
We obtain U «= S/M, where S is the total number of statements in the program. 

If the program is a procedure of a larger module, S is the number of state- 
ments in the procedure and M is the mean number of errors between statements 
of the module. Thus when calculated on a fractional basis, (3) becomes 

E(k)/U = ( I s. . (1-p. ,)/S. (4) 

ij ID ID ID 

It is important to use (4) when comparing detected errors from different size 
programs, since we would expect to find more errors in larger programs. 

The probability of traversing arc ij is computed by multiplying the 
probability of reaching node i by the branch probability for arc i j , 1/n, 
where n is the number of arcs emanating from node i. The probability of 
reaching node i is the sum of the traversal probabilities of arcs which enter 
node i . 



Navy Tactical Data System (NTDS) program listings were converted to 
directed graphs. After constructing the graphs, the numbers of nodes, arcs, 
paths and source statements were recorded. Simulation and analytical results 
were obtained for 44 procedures in two modules. The data were obtained in 
order to compare simulation and analytical results and to analyze complexity 
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measures (see Section IV) . Although any values of M could have been used for 
these purposes, the values were computed from S/U where S and U were 
actual numbers of module source statements and total reported errors, respectively. 
In the model, the distance between errors refers to statements executed and not 
statements on the program listing. M is equal to the model mean distance when 
module testing starts (all U errors are present) and all statements are 
executed once (S statements) . 

Appendix B contains the directed graphs and tables pertaining to the 
simulation and analytical solutions for 31 procedures of one module and 13 
procedures of a second module of the NTDS . The graphs and tables show numbers 
of nodes, arcs, paths and source statements. Simulation and analytical results 
are shown for mean number and mean fraction of errors detected and probability 
of arc traversal for every arc of every directed graph for a single input. 
Simulation results were obtained for 100 repetitions and 100 replications 
where a repetition is an error seeding and a replication is a path. One hundred 
replications were used for each of 100 repetitions. 

Validation Tests 

a. Mean Number of Errors Seeded. 

Errors are seeded in the simulation model with exponentially distributed 

distances (number of source statement) between errors. This is equivalent 

to a Poisson distribution of number of errors seeded per arc, with the mean 

number seeded being proportional to arc length. The total number of errors 

seeded in the directed graph is also Poisson distributed with mean S/M and 

1/2 

standard deviation (S/M) . Since the sample was N = 100 seedings , the 

normal approximation of the Poisson was used. Since the variance is known, 
the test statistic: 
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1/2 ““ 

Z = ( X - (S/M))/ (S/MN) , where X is the mean number of errors seeded 

over N seedings with M = 21 statements between errors . A two sided test was 
used with a = .05. Eight Module 1 NTDS procedures were tested for error 
seeding separately and independently of the results shown in Appendix B for 
error detection - 

Hq: y = S/M 

p / S/M 

Reject if |z| > 1.96 



The 


results of 


the hypothesis 


tests are 


shown in Table 








TABLE 
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Mean Error Seeding Tests 






Procedure 


£ 


S/M 


X 


(S/M) 


PI 


8 


10 


.476 


.550 


.690 


1.072 


14 


9 


.429 


.430 


.655 


.015 


25 


8 


.381 


.400 


.617 


.308 


34 


15 


.714 


.780 


.845 


.781 


39 


17 


.810 


.840 


.900 


.333 


47 


12 


.571 


.680 


.756 


1.442 


48 


13 


.619 


.700 


.787 


1.0292 


53 


11 


.524 


.630 


.724 


1.464 



Although would be accepted in each of the above tests , the simulation 

error seeding was consistently high. 

Another test involved a graph with a single input and a single exit node. 
The arc joining these nodes had a length of 10 and the parameter MEANER was set 
to 1 so that the expected number of seeded errors was 10. The subroutine SEED 
was called 1000 times to seed errors in this arc, and the mean number of errors 
seeded was 9.995. This test was conducted by running a 1000 inputs through the 
graph and since each input traverses the single arc the number of errors found 
is the same as the momber seeded ( Z = (10 - 9 . 9995) / (10 • 1000) = .00005). 
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b. Probability of Arc Traversal 



Traversals on a given arc or path are independent and the probability of 
traversal is constant on successive trials. The number of traversals in an 



arc ij is binomially distributed. The probability of arc traversal is 



and the relative frequency of traversal obtained from simulation is P . . , so 

I t 

that E(P. .) = P. . and V(P. .) = P. . (1-P. .)/N, where N is the number of 
13 13 13 13 13 

independent trails (100 replications x 100 repetitions = 10,000 trials). Since 
a normal approximation can be used when N is this large and the variance is 
known, the test statistic 



Z 





(l-P. . ) n] 
1: ^ 



was employed for a two sided test with a = .05. Eight procedures listed in 
Appendix B (different procedures than used in seeding tests) were randomly 
selected. A branch node of each of the eight procedures and its outgoing arcs 
were also randomly selected. 




H 



1 ' 



M ^ P, . 

13 



Reject if |zl > 1.96 



The results of the hypothesis tests are shown in Table III-2. The hypothesis 

would be accepted in each case. Simulation and analytical results are close, 
as can be seen by examining Appendix B. 
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TABLE III-2 



Probability of Arc Traversal Tests 



Module/ 

Procedure 


Arc 


1 

P. . 






Izl 


1/28 


4,5 


.1256 


.1250 


.331 


.181 


1/28 


4,6 


.1235 


.1250 


.331 


.453 


1/29 


2,3 


.5014 


.5000 


.500 


.280 


1/29 


2,6 


.4986 


.5000 


.500 


.280 


1/44 


5,6 


.2458 


.2500 


.433 


.970 


1/44 


5,k 


.2419 


.2500 


.433 


1.871 


1/49 


4,5 


.1260 


.1250 


.331 


.302 


1/49 


4,8 


.1232 


.1250 


.331 


.544 


1/60 


6,7 


.1225 


.1250 


.331 


.755 


1/60 


6,17 


.1213 


.1250 


.331 


1.118 


2/23 


2,3 


.4947 


.5000 


.500 


1.060 


2/23 


2,4 


.5053 


.5000 


.500 


1.060 


2/48 


5,6 


.1869 


.1875 


.390 


.154 


2/48 


5,7 


.1850 


.1875 


.390 


.641 


2/86 


5,12 


.2524 


.2500 


.433 


.554 


2/86 


5,20 


.2472 


.2500 


.433 


.647 
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c. Numbers of Arc and Path Traversals 



The branching mechanism was tested by including in the program a traversal 
counter which records the number of times each arc is traversed during a program 
run. The correct functioning of the counter was confirmed by obtaining detailed 
output (MOOT = 1) for several different graphs and manually confirming that 
the count corresponded to the detailed output. 

Several runs were made to test the actual traversal count against the expected 
number. Two specific tests are reported below: 

1. The graph with a single input node connected to each of 4 terminal nodes 

was used in a run with 1 input, 4000 replications, and 1 seeding. The expected 

number of traversals on each arc is 1000, and the traversal counter showed 994, 

1004, 1006, 996 as the observed frequencies. These values easily pass a chi square 

test at the 99% confidence level (y^ -.104, = 11.34). 

3^, .99 

2. A second test used a graph with node 1 connected to nodes 2 and 3, node 3 
connected to nodes 4 and 5, and node 5 connected to nodes 6 and 7. In addition 
10 inputs were used to test if the branching mechanism works for multiple inputs. 
The test used 100 replications and 2 seedings. The graph has a total of four 
paths from input to termination. The expected number of traversals for the path 
terminating at each node is shown below along with the actual number of traversals 
produced in this test. These values also pass the chi square test at the 99% 

confidence level (x^ = -419, 11.34). 

3, .99 

path ending at 2 4 6 7 

expected traversals 1000 500 250 250 

actual traversals 1011 491 253 245 
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IV. ANALYSIS OF COMPLEXITY 



One objective of the research was to identify complexity measures which 
could be used to estimate the difficulty of detecting errors in a program. 

If these measures could be identified, they would prove useful in program 
design and testing. Complexity measures would be used during program design 
to avoid structures which are difficult to test and during testing to allocate 
resources to testing on the basis of estimated difficulty of error detection. 

Four complexity measures were evaluated: numbers of nodes (Nn) , arcs, (Ma) , 

paths (Np) and source statements (S) . A brief explanation will be given of 
the significance of these measures as indices of difficulty of error detection. 
These measures were developed for 31 procedures of one NTDS module and 13 pro- 
cedures of another NTDS module. A procedure is a subset of a module. 

Node:^ 

Nodes can be categorized as follows : 

a. Start . Signifies program beginning. No entry and one or more exits. 

Terminal . Signifies program end. One or more entries and no exit, 
c. Branch . Single entry and multiple exits. 

« Multiple entries and single exit. 

e. Merge and Branch . Multiple entries and exits. 

f. Transfer . Single entry and exit. Commonly used to signify a call 

to a sub-procedure and to indicate the return point in the calling procedure. 

g. Dummy . A special case of f. An artificial node in the simulation 
for handling parallel arcs. 

h. Transient . Nodes which delineate the beginning and ending of 
sub-procedures. 
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As compared to a structure with a start node, terminal node and one arc 
connecting the two, the addition of nodes in categories c, d, and e will add 
to the number of paths and decrease the probability of reaching certain arcs. 
Thus, error detection is affected. Transfer nodes do not affect the structure 
of a program, but do add to its complexity because these nodes correspond to 
a point of transfer of control to a sub-procedure and a return to the calling 
procedure. Transient nodes were not counted in Nn because transient nodes 
are part of a sub-procedure; however, the transient nodes were necessary in 
order to complete certain paths in the directed graph. Dummy nodes were not 
counted as part of Nn because they were used only to specify a directed 
graph in the model and did nest contribute to program complexity. All other 
nodes were counted. It would be useful in future work to develop an additional 
measure consisting only of nodes involving branching, since the presence of 
these nodes may cause the probabilities of arc traversal in the outgoing arcs 
to be less than the probabilities in incoming arcs, thus increasing the 
difficulty of error detection in paths which contain the outgoing arcs. 

Arcs 

Transient arcs are arcs contained in sub-procedures. These arcs were not 
counted in Na. However, these arcs were needed to complete certain paths in 
the directed graphs. An entire sub-procedure was represented by two nodes 
(start and end nodes) and one arc. This was done because it was infeasible 
to represent in the model an entire module of approximately 2000 source 
statements and many interconnected procedures. Secondly, the point of view 
was adopted that sub-procedures called by procedures have been checked out 
and any errors detected reside in the main procedure. Thus the sub-procedure 
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was treated as a zero source statement (no errors were seeded) arc. This 
treatment corresponds to viewing the test of a procedure as a unit test 
in which it is assumed that interacting units (sub-procedures ) are working; 
the focus is on finding errors in the main procedure. An important facet 
of testing is the integration test in which the focus is on the inter- 
actions between units. The model could be employed in this vein by conceiving 
of each procedure as an arc and the nodes as branch points to entire 
procedures. It was not necessary to depict transient nodes and arcs, since 
these components do not affect the error finding mechanism in the 
model. However, it was considered important to document the actual program 
structure rather than the structure required by the model. 

Dxjimmy arcs were not counted in Na because they were used only to 
represent parallel arcs in the model and did not contribute to program 
complexity. 

An increase in the number of arcs increases the number of paths and, 
hence, makes error finding more difficult. 

Paths 

We define a path as a series of connected nodes and arcs which begins 
with a start node and ends with a teminal node. A definitional problem 
arises in the case of paths which contain cycles. A graph containing 
cycles has an infinite number of paths. However, for our purposes, 
traversals in cycles were counted the minimum number of times. A cycle 
contained in a path was traversed one time. This treatment of paths is 
consistent with the model assumption that all errors in an arc are detected 
on the first traversal. Cycles in the model do not represent DO loops 
in a program, where the program is forced to iterate in a loop a 
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specified number of times. Rather, a cycle contains nodes and arcs which 
may be revisited. 

As the number of paths in a program increases, error finding becomes 
more difficult because the number of areas in the program which must be 
searched increases. It should be noted that number of paths is not 
independent of number of nodes and arcs; number of paths is a function 
of number of nodes and arcs. 

Source Statements 

The number of source statements S will affect error detection in 
the model because an assumption of the model is that the number of original 
errors in a program is proportional to the number of source statements. 

Based on this assumption, error detection would increase as program size 
increases, all other factors (number of paths) being equal. In order to 
not mask the effect of other factors, the number of errors detected on 
the first input E{1) , as a fraction of the original number of errors U, 
is used instead of E(l). The relationship between error detection and S 
is complex because error detection depends on both U (a function of S) 
and structural properties (paths) . 

Structural properties are not independent of number of source statements. 
Since few programs are written as one line of code or a single arc, the 
number of nodes, arcs and paths will, in general, increase with number of 
source statements. 

Analysis of Complexity Measures 

Values of Nn, Na, Np, S and E(l)/U (analytical solution) are 
tabulated in Table IV-1. Plots of E(l)/U versus Nn, Na, Np and S 
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are shown in Figures IV-1 to IV-4, respectively. Points are plotted for 
both Module 1 and Module 2. Since the data were shown to be highly non- 
linear when plotted on a linear scale, the data were plotted first on 
semi-log and then on log-log scales (Figures IV-1 to IV-4) to see whether 
a straight line would emerge indicative of a non-linear relationship. 

As seen in the figures, the data are still scattered on a log-log scale. 
In many cases there are multiple values of errors detected for a given 
value of complexity. This result suggests that error detection is a 
function of multiple complexity measures. However, lack of independence 
among variables makes the identification of a multi-variate relationship 
difficult. Other measures, such as path length, connectivity and 
reachability, may prove more illuminating as indices of error detection. 

In order to more clearly indicate the relationship between error 
detection and a single complexity measure, multiple values of E(l)/U 
were averaged (for three or more values) and plotted in Figures IV-5 
and IV-6 for paths and nodes, respectively. These curves provide a 
better indication of the inverse relationship between error detection 
and complexity. However, the reduced sample which results from averaging 
is inadequate for fitting an equation to the data. 
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TABLE IV-1 



COMPLEXITY MEASURES 



Module/ 

Procedure 


Nn 


Na 


Np 


s 


E(l)/1 


1/2 


12 


21 


26 


37 


.1700 


1/8 


8 


11 


3 


10 


.6000 


1/11 


6 


7 


3 


8 


.5938 


1/14 


4 


5 


4 


9 


.7222 


1/19 


17 


25 


7 


22 


.3295 


1/22 


18 


26 


11 


30 


.3458 


1/25 


8 


10 


2 


8 


.6875 


1/28 


12 


16 


4 


24 


.6145 


1/29 


20 


28 


13 


47 


.6463 


1/30 


7 


10 


5 


10 


.3563 


1/34 


11 


15 


3 


15 


.9167 


1/35 


12 


16 


3 


11 


.8636 


1/36 


17 


24 


3 


31 


.4032 


1/39 


15 


24 


10 


17 


.3526 


1/44 


15 


24 


7 


21 


.4256 


1/47 


12 


16 


4 


12 


.7500 


‘ 1/48 


14 


21 


7 


13 


.6538 


1/49 


13 


19 


7 


19 


.2829 


1/53 


11 


18 


9 


11 


.4531 


1/57 


22 


31 


12 


26 


,2726 


1/60 


22 


34 


16 


24 


.3317 


1/75 


17 


24 


8 


20 


.6800 


1/76 


13 


18 


5 


19 


.4803 


1/77 


10 


16 


9 


10 


.6000 


1/79 


19 


28 


3 


23 


.3804 


1/81 


6 


8 


3 


7 


.5000 


1/87 


14 


20 


6 


25 


.4700 


1/91 


18 


28 


9 


14 


,2514 


1/92 


4 


5 


3 


9 


.4444 


1/93 


23 


33 


12 


34 


.2289 


1/95 


16 


26 


10 


19 


.2325 
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TABLE IV- 1 (cont*d) 



COMPLEXITY MEASURES 



Module/ 



Procedure 


Nn 


Na 


Np 


S 


E(l)/U 


2/15 


9 


12 


5 


10 


.47 50 


2/23 


8 


10 


3 


12 


. 6667 


2/41 


9 


13 


12 


13 


.5192 


2/46 


23 


32 


18 


32 


.3212 


2/47 


22 


34 


36 


33 


.2060 


2/48 


12 


18 


14 


17 


.4651 


2/79 


9 


12 


5 


11 


.5455 


2/86 


20 


29 


6 


33 


.3775 


2/90 


11 


17 


8 


23 


.2446 


2/99 


19 


27 


10 


23 


.4620 


2/122 


12 


18 


6 


20 


.4313 


2/137 


10 


15 


11 


23 


.3505 


2/149 


16 


24 


9 


35 


.3964 


Legend 


Nn: number of 


nodes 


, with dummy 


nodes (nodes 


associated 


with dummy 


parallel 


arcs) 


and transient 


nodes (nodes 


assoc iated 


with call^ 



procedures) eliminated, 

Na: number of arcs, with dummy arcs and transient arcs (arcs 

associated with called procedures) eliminated. 

Np: number of paths, with paths involving cycles counted minimum 

number of times. 

S: number of source language statements. 



E(l)/U: expected fraction errors detected on first imput obtained with 

analytical model. 



IV- 7 



46 7283 



IV-8 




IV- 9 




2 X 27 CYCLES 



IV- 10 





00 



O 



I®' 

103 



LOGARITHMIC 46 7263 

■’'’iS 2 X 2.7 CYCLtS M»Dt im U. S a. 

KEUKFfel. a ESSER CO, 



IV-12 





n/(i)3 









00 



m 



CO 



CN 



V. SUMMARY 



This report covered the following areas: 

. Description of the modified error simulation model 
. Validation tests of the simulation model 
. Analysis of program complexity measures 

A description of the revised simulation model was presented in Section 

II. The data input format was described in Section II-4. A flowchart 
appears in Appendix A, 

Validation tests on the simulation model were described in Section 

III. Some of the tests used data shown in Appendix 3. These data show: 
directed graphs, properties of the directed graphs, simulation solutions 
and analytical solutions for 44 NTDS procedures. Validation tests were 
conducted for error seeding and arc/path traversal. All hypothesis 
(validation) tests were passed. However, simulation results were 
consistently higher than analytical results. 

Program complexity measures were analyzed in Section IV. Four 
complexity measures — numbers of nodes, arcs, paths and source statements — 
were plotted against fraction expected detected errors obtained from the 
analytical solution. The data were obtained from the NTDS procedures of 
Appendix B. Although the direction of the plots (error detection inversely 
related to complexity measures) was as expected, considerable variability in 
the data were exhibited. The variability indicated error detection was a 
function of several complexity measures instead of one. There were multiple 
values of error detection for many values of complexity measure. When these 
values were averaged the effect of a single complexity measure became 
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clearer. However, the reduced sample size made quantitative analysis 
infeasible. Although the four factors are potentially useful as measures 
of complexity, no single factor stood out as being a major determinant of 
error detection. 
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APPENDIX A 



FLOW CHART FOR ERROR 

SIMULATION PROGRAM (MAIN) , 
AMD SUBROUTINE (SEED) 
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Select initial values for 
random number generators 

IX= 

IW= 

IZ= 

(any integer between 1 and 2147483647) 




A-3 



© 





A-4 



© 



IREP=0 

replication counter 




IREP=IREP+1 






/ 






NODE=L 



A-6 



© 




no 






yes 




_tL 



113 



ISEEDd ,J) =SVSEED(I ,J) 
for all I,J 
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Do 666 , this page, for 
each input 1,...,MINPUT 




yes 




1, . . . ,MINPUT 





7776 




If NREPET=1 


(# seedings) 


T 

1 


or NUMOUTt^I 


(# replications) 


1 


(or both) , GO TO 7766 
Otherwise, continue. 


-J 


^ _ '1 


f 


VAR1=SVSQR( 


) -NREPET*TAVE*TAVE 






NREPET-1 


1 


SDl = VARl** 


. 5 


i 



JL 



Write "STD DEV OVER SEEDING WITH ONE REPLIC=' 
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£ 

Write "I J NBR OF TRAVERSALS" 
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I 
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APPENDIX B 



DIRECTED GRAPHS, SMULATION AND ANALYTICAL RESULTS 

1 . Notation used in Appendix B 
i : node i 

j : node j 

P \ : relative frequency of traversing arc ij one or more times 

(simulation model) 

P^ j : probability of traversing arc ij one or more times (analytical 

model) . 

S. number of source statements in arc ij . 

E(l) = P. . S. ./M; expected number of errors detected on first input, 
ij 

obtained from analytical model, where M is the mean number of 

source statements between errors (M = 21 for Module 1 and M = 51 

for Module 2) . 

/ 

U = S/M: expected number of errors in a program, where S is the niimber 

of source statements in a program. 

E' (1)/U = ^ P. .S. ./S: expected fraction of number of errors detected on 
ij 

first input. 

E'(l): mean number of errors detected on first input obtained from 

simulation model. 

E'(l)/U = (e‘( 1)/S)M: mean fraction of number of errors detected on 

first input (given as a percentage at the bottom of directed 
graphs) . 

*: indicates that P.\ has no meaning because ratio of number of traversals 

in arc ij to total number of input traversals is greater than one, 
because arc ij is part of a cycle. 
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2, Notes, 



o The number of source statements (x) and number of machine instructions (] 
in an arc are indicated by (x/y) alongside the arc. No number or zero 
means there are zero statements/instructions in an arc. (These are trans: 
of control arcs) . In some cases the number of machine instructions was 
not available. 

o Since the simulation model does not accomodate parallel arcs, a dummy 
node was inserted in each parallel arc. 

Nodes associated with sub-procedures (entry and exit nodes) are designat< 
by letters. The entire sub-procedure is indicated by a dotted line 
and is counted as one arc. 
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Hcduls: 1 



Procedu r e No . : 2 




Nuiii:eEofncc€£: 14 

Nunc€Ecfarc£: 23 

Nurrfcecof paths: 26 

Nunter ci sccrce stnts.: 37 

A '/ •= r a g a = t r c r found: 0.3144 

tercentace eiicrs found: 17. c4 
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Module 1 



Procedure 2 



100 Replications 100 Repetitions 



i 


j 


"■ij 




s . . 
13 


1 


2 


1.0000 


1.0000 


4 


2 


3 


.4948 


.5000 


1 


2 


14 


.5052 


.5000 


0 


3 


4 


.2480 


.2500 


2 


3 


8 


.2468 


.2500 


0 


4 


5 


.1250 


.1250 


3 


4 


14 


.1230 


.1250 


0 


5 


6 


.0645 


.0625 


4 


5 


8 


.0605 


.0625 


0 


6 


7 


.0315 


.0313 


0 


6 


8 


.0330 


.0313 


4 


7 


8 


.0315 


.0313 


0 


8 


9 


.1860 


.1875 


1 


8 


14 


.1858 


.1875 


0 


9 


10 


.0949 


.0938 


1 


9 


14 


.0911 


.0938 


0 


10 


11 


.0475 


.0469 


1 


10 


14 


.0474 


.0469 


0 


11 


12 


.0233 


.0234 


2 


11 


14 


.0242 


.0234 


0 


12 


13 


.0101 


.0117 


0 


12 


14 


.0132 


.0117 


14 


13 

y p. .s. . = 6.2890 
..13 13 
13 

Ed) = .2995 
E(l)/U = .1700 


14 


.0101 


.0117 


0 

37 
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licdule: 1 



Procedure liO. i 8 




il u 1 1 e r 


0 f 


ncces: 


13 


fl u m h e r 


o £ 


ar cr : 


14 


^i u n h e r 


c f 


paths: 


n 

O 


il u IT b € r 


c£ 


serree stats.: 


10 


Average 


er 


r cr found : 


0.2523 


L- : r c e n t 


aqe 


errors round: 


52.93 
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Module 1 



Procedure 8 



100 Replications 100 Repetitions 





i 


^ ij 




S . . 

ID 


1 


2 


1.0000 


1.0000 


3 


2 


3 


.4947 


.5000 


3 


2 


9 


.5053 


.5000 


0 


3 


A 


.4947 


.5000 


0 


4 


5 


.4947 


.5000 


1 


5 


C 


.4947 


.5000 


0 


6 


7 


.4947 


.5000 


1 


7 


8 


.2497 


.2500 


0 


7 


9 


.2450 


.2500 


2 


8 


9 


.2497 


.2500 


0 


A 


B 


.4947 


. 5000 


0 


B 


4 


.4947 


.5000 


0 


C 


D 


.4947 


. 5000 


0 


D 


6 


.4947 


.5000 












10 


y p. . s . . = 6. 0000 

••ID ID 

ij 










Ed) = .2857 










E(l)/U = .6000 
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tlcdule: 1 



Procedure No . : 11 



Nuahsr of 
N u a t € r of 
N Ulcer of 
Number of 
A 7 e r a g e e 
Perce ntac 




nodes: 6 

arcs: 7 

paths: 3 

sc tree stats.: 8 

rrer feund: 0.1974 

e errors found: 51.82 
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Module 1 



Procedure 11 



100 Replications 100 Repetitions 



i 


i 






S^j 


1 


2 


1.0000 


1.0000 


2 


2 


3 


.4947 


.5000 


1 


2 


4 


.5053 


.5000 


0 


3 


4 


.2497 


.2500 


1 


3 


6 


.2450 


.2500 


0 


4 


5 


.7550 


.7500 


2 


6 


5 


.2450 


.2500 


2 



8 



I P. .s. . 
13 13 



4.75 



Ed) = .2262 
E(l)/U = .5938 
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McGule; 1 



Procedure No.: 14 




Muater of nodes: 

Nurher of arcs: 

Nucher of paths: 

Nuiiter cf scrrce stuts.: 
Average errcr found: 
Eercenrace errors rcuni: 



6 

7 



4 

9 

0.2585 

O ^ '-r 
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Module 1 



Procedure 14 



100 Replications 100 Repetitions 



i 


i 


"ij 




S . . 


1 


2 


1.0000 


1.0000 


4 


2 


3 


.4983 


.5000 


0 


2 


4 


.5017 


.5000 


2 


3 


4 


.4983 


.5000 


0 


4 


5 


.4994 


. 5000 


0 


4 


6 


.5006 


.5000 


3 


5 


6 


.4994 


.5000 


0 










9 


I P. -s. . 
? . ij ij 

ID 


= 6.5000 








E(l) = . 


3095 








E(l)/U = 


.7222 
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ilcdule: 1 



Proceduce Jio . : 



19 




Number of 


nodes; 


19 


N u n b e r c f 


a Cos : 


26 


Nudber of 


paths: 


7 


N usher of 


s o ubce stats.; 


22 


Average errgb found; 


0.2385 


Percentac 


e errors found; 


27.54 
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Module 1 Procedure 19 





100 


Replications 100 


Repetitions 


i 


1 


p' • • 




1 


2 


1.0000 


1.0000 


2 


3 


.4948 


.5000 


2 


19 


.5052 


.5000 


3 


4 


* 


.2500 


3 


5 


.4948 


.5000 


4 


3 


* 


.2500 


5 


A 


.4948 


.5000 


6 


7 


* 


.1667 


7 


8 


* 


.0909 


7 


14 


* 


.1000 


8 


A 


* 


.0909 


9 


10 


* 


.1538 


10 


A 


* 


.1538 


11 


12 


* 


.1538 


12 


A 


* 


.1538 


13 


19 


.2206 


.2222 


14 


15 


* 


.0500 


14 


19 


.0560 


.0556 


15 


A 


* 


.0500 


16 


19 


.2182 


.2222 


A 


B 


* 


.5000 


B 


6 


* 


.1667 



3 

1 

0 

0 

4 

0 

0 

1 

3 

1 

0 

2 

0 

2 

0 

1 

3 

0 

0 

1 

0 

0 
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B 


9 


* 


.1538 


0 


B 


11 


* 


.1538 


0 


B 


13 


.2206 


.2222 


0 


B 


16 


.2182 


.2222 


0 



22 



Z p. . s. . = 7.2490 
ID ID 

E(l) = .3452 
E(l)/u = .3295 



B-13 



Mcdule: 1 



Procedure No.: 



22 




iJuater of ncdes: 
flutt’cer cf arcs: 

Nuafcsr cf faths: 

Nuiifcer cf scLice stmts.: 
Average errcr found: 
Percentage errors found: 



25 
30 
1 1 
30 

0.4105 

23.70 



B-14 



Module 1 



Procedure 22 



100 Replications 100 Repetitions 



i 




i 


"i: 


"ij 


s . . 
13 


1 




2 


1.0000 


1.0000 


4 


2 




3 


.4950 


. 5000 


4 


2 




19 


.5050 


. 5000 


0 


3 




4 


.2490 


.2500 


0 


3 




5 


. 2460 


.2500 


3 


4 




5 


.2490 


.2500 


0 


5 




6 


.1280 


.1250 


1 


5 




7 


.1179 


.1250 


1 


5 




8 


.1234 


.1250 


1 


5 




9 


.1257 


.1250 


1 


6 




16 


.1280 


.1250 


2 


7 




15 


.1179 


.1250 


2 


8 




10 


.1885 


.1875 


3 


9 




14 


.1257 


.1250 


1 


10 




A 


.1885 


.1875 


0 


11 




12 


.1885 


.1875 


2 


12 




C 


.1885 


. 1875 


0 


13 




18 


.1885 


.1875 


0 


14 




3 


.0651 


.0625 


0 


14 




15 


.0606 


.0625 


1 


15 




16 


.1785 


.1875 


1 


16 




17 


.3065 


. 3125 


1 


17 




E 


.3065 


. 3125 


0 


18 




19 


.4950 


.5000 


2 


A 




B 


.1885 


.1875 


0 


B 




11 


.1885 


.1875 


0 


C 




D 


.1885 


. 1875 


0 


D 




13 


.1885 


.1875 


0 


E 




F 


.3065 


. 3125 


0 


F 




18 


. 3065 


.3125 


_0 












30 


y p. .s . . = 10 
••13 13 
13 


.375 










E(l) = .4940 












E(l)/U = 


.3458 











B-15 



Mcculs: 1 



Procedure No . : 



25 



1/1 




Nuater of ncdes: 12 

Nurter cf arcs; 12 

Nunhsrofpatbs; 2 

Nuarer cf source stats.: 3 

Average errcr found: 0.2324 

Percentage errors icund; 6 1.01 



B-16 



Module 1 



Procedure 25 



100 Replications 100 Repetitions 



1 

2 

3 

3 

4 

5 

6 
7 
A 
B 
C 
D 



1 

2 

3 

4 
8 
A 
6 
C 
2 
B 

5 
D 
7 




1.0000 



★ 



I P. .s. . 

i: 



5.5 



Ed) = 
E(l)/U 



. 2619 

.6875 



1.0000 1 

1.0000 2 

. 5000 2 

. 5000 0 

. 5000 0 

.5000 2 

.5000 0 

. 5000 1 

.5000 0 

. 5000 0 

. 5000 0 

. 5000 0 

8 



B-17 



tied ule 



1 



Procedure No. : 28 




N u ii 1 6 r 


o f 


nodes: 


17 


N U E t € r 


Cl 


arcs; 


19 


N u 0 t e r 


c f 


paths: 


4 


N u D i: e r 


cf 


sccrce stats.: 


24 


A V s E a g 


e er 


rcr found: 


0.6400 


E e r c e a 


ta c e 


e irer 3 found; 


56. 00 



3-18 



Module 1 



Procedure 28 



100 Replications 100 Repetitions 



i 


i 


p' . . 
13 




s , . 
13 


1 


2 


1.0000 


1.0000 


5 


2 


3 


.4975 


. 5000 


2 


2 


11 


.5025 


. 5000 


0 


3 


4 


.2491 


.2500 


6 


3 


12 


.2484 


. 2500 


0 


4 


5 


.1256 


. 1250 


0 


4 


6 


.1235 


.1250 


0 


5 


6 


.1256 


.1250 


0 


6 


7 


.7516 


.7500 


5 


7 


A 


.7516 


.7500 


0 


8 


9 


. 7516 


.7500 


1 


9 


C 


.7516 


.7500 


0 


10 


13 


.7516 


.7500 


1 


11 


6 


. 5025 


. 5000 


4 


12 


13 


. 2484 


. 2500 


0 


A 


B 


.7516 


.7500 


0 


B 


8 


.7516 


. 7500 


0 


C 


D 


.7516 


.7500 


0 


D 


10 


.7516 


.7500 


_0 










24 


J p. ,s. . = 14. 
• • 13 13 
13 


. 75 








E(l) = .7024 










E(l)/U = 


6145 









B-19 



Module: 1 Procedure No.: 29 




B-20 




Module 1 Procedure No. 29 



100 Replications 100 Repetitions 



1 


1 


p • 
i: 


j 


s . . 


1 


2 


1 . 0000 


1 . 0000 


4 


2 


3 


. 5014 


. 5000 




2 


6 


. 4986 


. 5000 


0 


3 


4 


. 2499 


.2500 


3 


3 


20 


. 2515 


.2500 


1 


4 


A 


. 2499 


. 2500 


0 


5 


7 


. 2499 


. 2500 


3 


6 


7 


.4986 


. 5000 


1 


7 


8 


. 7485 


. 7500 


15 


8 


C 


. 7485 


. 7500 


0 


9 


10 


. 7485 


. 7500 


5 


10 


11 


.3682 


. 3750 


2 


10 


13 


. 3793 


.3750 


0 


11 


E 


. 3692 


.3750 


0 


12 


13 


. 3692 


.3750 


3 


13 


14 


.7485 


. 7500 


2 


14 


15 


* 


.3750 


0 


14 


16 


.748 5 


. 7500 


4 


15 


14 


* 


. 3 7 50 


0 


16 


G 


.7485 


. 7500 


0 


17 


18 


. 7445 


. 7500 


2 


18 


G 


.7445 


. 7500 


0 


19 


20 


.7485 


. 7 500 


1 


A 


B 


. 2499 


. 2500 


0 


B 


5 


. 2499 


.2500 


0 


C 


D 


. 7485 


.7 500 


0 


D 


9 


. 7485 


. 7500 


0 


E 


F 


.3692 


.3750 


0 


F 


12 


. 3692 


. 3750 


0 


G 


H 


* 


.7500 


0 


H 


17 


.7445 


.7 500 


0 


H 


19 


. 7485 


.7500 


0 



47 



B-21 



Module 1 



Procedure No. 29 



(continued) 



p..S.. = 30.375 

ID ID ID 

E(l) = 1.4464 

E(l)/U = .6463 



/ 



B-22 



iicdule: 1 



Procedure lie . : 30 




lluKtez 


o f 


ncces; 


7 


i; u I fc e i: 


c f 


arcs : 


10 


Nu I her 


c f 


paths: 


5 


i'l u r:- h 6 r 


cf 


sccice stDts.: 


10 


Aver a g 


9 


rcr fcunc: 


0 . 164 9 


ic€n tac€ 


e I r c r s round: 


34 . 63 



B-23 



Module 1 



Procedure 30 



100 Replications 100 Repetitions 



i 


j 


P’ . . 
ID 




s . . 


1 


2 


1.0000 


1.0000 


2 


2 


3 


.4943 


.5000 


1 


2 


7 


.5057 


.5000 


0 


3 


4 


.2500 


.2500 


1 


3 


7 


.2443 


. 2500 


0 


4 


5 


.1276 


.1250 


1 


4 


6 


.1224 


.1250 


0 


5 


6 


.0638 


.0625 


0 


5 


7 


.0638 


.0625 


2 


6 


7 


.1862 


.1875 





10 



J p. .s. . = 3.5625 
1 ] 

E(l) = .1696 
E(l)/U = .3563 



B-24 



Module; 1 



Procedure IJo.: 3^4 




N u n £ 6 r 


c £ 


ncces: 


16 


N u a i; 6 c 


CI 


ar cs : 


18 


Mu ere r 


o f 


paths; 


3 


U u c 1 6 r 


cf 


source stnts.; 


1 5 


A 7 s r a g 


o 8 


xrcr found; 


0 . 5465 


PcECsntac 


e errors found; 


76.51 



B-25 



Module 1 



Procedure No. 34 



100 Replications 100 Repetitions 



1 


1 


" ij 




s . . 


1 


2 


1 .0000 


1 . 0000 


2 


2 


A 


1.0000 


1. 0000 


0 


3 


4 


1. 0000 


1 .0000 


1 


4 


5 


. 4994 


. 5000 


1 


4 


7 


. 5006 


. 5000 


0 


5 


6 


. 2475 


. 2500 


0 


5 


7 


.2519 


. 2500 


1 


6 


7 


. 2475 


.2500 


0 


7 


8 


1 . 0000 


1 .0000 


5 


8 


C 


1 . 0000 


1 .0000 


0 


9 


10 


★ 


1 . 0000 


2 


10 


C 


* 


1 . 0000 


0 


11 


12 


1 . 0000 


1 .0000 


3 


C 


D 


* 


1 . 0000 


0 


D 


9 


★ 


1 . 0000 


0 


D 


11 


1 .0000 


1 .0000 


0 


A 


B 


1 . 0000 


1 .0000 


0 


B 


3 


1 .0000 


1 .0000 


0 










15 


I 

. . P . . s . . 
ID ID ID 


= 13. 


75 






E(l) = 


.6548 








E (1) /U = 


. 9167 









B-26 



ilcdule; 1 



Procedure No. : 



35 




i: u a b e E 


0 f 


cedes : 


14 


N u K h e r 


c f 


arcs; 


17 


K u a t e r 


0 f 


p a t bs : 


3 


2i u E b e r 


of 


scuEce stats.: 


1 1 


A V e r a q 


e errcr found: 


0.3576 


P ? r c e n 




s eircrs founf: 


63.27 



3-27 



Module 1 



procedure No. 35 



IQQ Replications 



IQQ Repetitions 



i 


j 


'■Is 




s . . 
ID 


1 


2 


1 . 0000 


1 .0000 


4 


2 


A 


1. 0000 


1 . 0000 


0 


3 


4 


* 


1 , 0000 


1 


4 


5 


. 5075 


. 5000 


1 


4 


6 


.5055 


. 5000 


0 


5 


6 


,2563 


. 2500 


0 


5 


9 


.2512 


. 2500 


0 


6 


7 


. 7618 


.7500 


4 


7 


A 


.7618 


. 7500 


0 


8 


9 


* 


1 . 0000 


0 


9 


10 


* 


1 . 0000 


0 


10 


A 


* 


1 , 0000 


0 


11 


12 


1. 0000 


1 . 0000 


1 


A 


B 


* 


1 . 0000 


0 


B 


3 


* 


1. 0000 


0 


B 


8 


* 


1 . 0000 


0 


B 


11 


1 . 0000 


1. 0000 


0 










11 


I 

. . p . . s 
ID ID 


i j 


9 . 5 






E(l) = 


• 


4524 






E (1) /U 


= 


8636 







B-28 



Hcdule: 1 



Procedure lio.: 36 




N u a t e r 


Of 


ncces: 


21 


Nue cer 


c f 


arcs : 


26 


N u a t e r 


of 


paths: 


3 


N u a b 6 r 


cf 


scrrce stats.: 


31 


A ve rag 




r r c r found : 


0.5203 


P e r c e a 


tag 


s errors found; 


35.25 



B-29 



Module 1 Procedure 36 



100 Replications 100 Repetitions 







p • • 
ID 




s . . 


1 


2 


1.0000 


1.0000 


2 


2 


3 


.4964 


.5000 


12 


2 


17 


.5036 


.5000 


0 


3 


4 


.2449 


.2500 


3 


3 


6 


.2515 


.2500 


0 


4 


A 


,2 449 


.2500 


0 


5 


6 


* 


.2500 


0 


6 


7 


* 


.2500 


4 


7 


A 


* 


.2500 


0 


8 


9 


* 


.2500 

/ 


2 


9 


A 


* 


.2500 


0 


10 


11 


* 


.2500 


2 


11 


A 


* 


.2500 


0 


12 


13 


* 


.2500 


1 


13 


C 


* 


.2500 


0 


14 


15 


★ 


.2500 


4 


15 


A 


★ 


.2500 


0 


16 


17 


.4964 


.5000 


1 


A 


B 


★ 


.5000 


0 


B 


5 


* 


.2500 


0 


B 


8 


* 


.2500 


0 


B 


10 


* 


.2500 


0 


B 


12 


* 


.2500 


0 


B 


16 


.4964 


.5000 


0 



B-30 



C D 

D 14 



.2500 0 

.2500 0 

31 



Z P. . S. . = 12.5000 

E(l) = .5952 
E (l)/u = .4032 



B-31 



clcdule: 1 



Procedure Mo.: 39 




Muaterofncces: 17 

Musterofarcs: 25 

Muuberofpaths: 10 

tiuBbsr cf scLrca stats.: 17 

Average errcr found: 0.2637 

Percenrace errors found; 32.57 



B-32 



Module 1 



Procedure 39 



100 Replications 100 Repetitions 



i 


1 


ij 


ID 




1 


2 


1.0000 


1.0000 


4 


2 


3 


.4954 


.5000 


1 


2 


14 


. 5046 


.5000 


0 


3 


4 


.2490 


.2500 


1 


3 


14 


.2464 


.2500 


0 


4 


5 


.1258 


.1250 


1 


4 


14 


,1232 


.1250 


0 


5 


6 


.0631 


.0625 


1 


5 


14 


.0627 


.0625 


0 


6 


7 


.0299 


.0313 


1 


6 


14 


.0332 


.0313 


0 


7 


8 


,0162 


.0156 


1 


7 


14 


,0137 


.0156 


0 


8 


9 


.0090 


,0078 


1 


8 


13 


.0072 


.0078 


1 


9 


10 


,0040 


.0039 


1 


9 


11 


.0050 


. 0039 


0 


10 


11 


,0040 


.0039 


1 


11 


A 


,0123 


.0117 


0 


12 


15 


.0123 


.0117 


1 


14 


15 


.9877 


. 9883 


1 


13 


11 


.0033 


.0039 


1 


13 


14 


.0039 


.0039 


0 


A 


B 


.0123 


.0117 


0 


B 


12 


.0123 


.0117 


_0 



17 



p. .s. . 
; ID ID 



6,0117 



E(l) = .2863 



E(l)/U = . 3536 



B-33 



Module: 1 



Procedure Iio.: ^t4 




fiuttfcer of nodes: 27 

N umber of arcs; 30 

Number of paths; 7 

Number cf source stats.: 21 

Average error fcunc: 0.3554 

Percentage errors found: 35.54 



b-34 



i 

1 

2 

3 

4 

4 

5 

5 

6 

6 

7 

8 

9 

9 

10 

11 

12 

13 

14 

G 

H 

A 

B 

C 

D 



Module 1 Procedure 44 
100 Replications 100 Repetitions 



1 


p' . . 






2 


1.0000 


1.0000 


2 


A 


1.0000 


1.0000 


0 


4 


1.0000 


1.0000 


1 


5 


.4877 


.5000 


1 


6 


.5123 


.5000 


0 


6 


.2458 


.2500 


1 


K 


.2419 


.2500 


0 


7 


.3846 


.3750 


7 


C 


.3735 


.3750 


0 


E 


.3946 


.3750 


0 


9 


.3846 


.3750 


^ 1 


10 


.1952 


.1875 


4 


14 


.1894 


.1875 


0 


G 


.1952 


.1875 


0 


12 


.1952 


.1875 


2 


I 


,4371 


.4375 


0 


14 


.4371 


.4375 


1 


27 


.6265 


.6250 


1 


H 


.1952 


.1875 


0 


11 


. 1952 


.1875 


0 


B 


1.0000 


1.0000 


0 


3 


1.0000 


1.0000 


0 


D 


.3735 


.3750 


0 


27 


.3735 


.3750 


0 



B-35 



E 


F 


.3735 


.3750 


0 


F 


8 


.3846 


.3750 


0 


I 


J 


.4371 


.4375 


0 


J 


13 


.4371 


.4375 


0 


K 


L 


.2419 


.2500 


0 


L 


12 


.2419 


.2500 


0 



21 



Z p. . s. . = 8.9375 
ID ij 

E(l) = .4256 
E(l)/u = .4256 



B-36 



Module: 1 



Procedure No.: 47 




N u s b € r of 


nodes: 


19 


N u ffi b e r of 


arcs: 


20 


Nuuber of 


paths: 


4 


Nuaber of 


source stats,: 


12 


Average e 


rror found: 


0.4231 


Per cent ao 


e errors found; 


74.04 



b-37 




Module 1 Procedure 47 



100 Replications 100 Repetitions 



i 


j 




p' . . 






1 


2 




1.0000 


1.0000 


3 


2 


A 




1.0000 


1.0000 


0 


3 


4 




1.0000 


1.0000 


1 


4 


5 




.4913 


.5000 


0 


4 


6 




.5017 


.5000 


1 


5 


6 




.4983 


.5000 


3 


6 


7 




1.0000 


1.0000 


0 


7 


C 




1.0000 


1.0000 


1 


8 


9 




1.0000 


1.0000 


1 


9 


10 




.4994 


.5000 


0 


9 


12 




.5006 


.5000 


0 


10 


E 




.4994 


.5000 


0 


11 


12 




.4994 


.5000 


2 


12 


19 




1.0000 


1.0000 


0 


A 


B 




1.0000 


1.0000 


0 


B 


3 




loOOOO 


1.0000 


0 


C 


D 




1.0000 


1.0000 


0 


D 


8 




i.o5oo 


1.0000 


0 


E 


F 




.4994 


.5000 


0 


F 


11 




.4994 


.5000 


0 












12 


Z p , . s . , 
13 13 


= 


9.0000 








E(l) 


= 


.4286 








E(l)/u 


= 


.7500 









B-38 



tied ule 



1 



Procedure No.: 48 




Hunter of 


nodes: 


23 


Nuirber cf 


arcs: 


26 


Hunter of 


paths: 


7 


Huoher cf 


s c r rce s tn ts . : 


13 


Average errer found: 


0.3287 


f cr c€nrd c 


e errors found: 


53. 1 C 



B-39 




Module 1 Procedure 48 





100 


Replications 100 


Repetitions 








p' . . 




s . . 


i 


i 








1 


2 


1.0000 


1.0000 


2 


2 


3 


.4943 


.5000 


1 


2 


8 


.5057 


.5000 


0 


3 


4 


.2520 


.2500 


1 


3 


G 


.2423 


.2500 


0 


4 


5 


.12 88 


.1250 


0 


4 


6 


.1232 


.1250 


1 


5 


6 


.1288 


.1250 


0 


6 


A 


.2520 


.2500 


0 


7 


8 


.2520 


.2500 


0 


8 


9 


.7577 


.7500 


" 4 


9 


C 


.7577 


.7500 


0 


10 


11 


.7577 


. 7500 


1 


11 


12 


.3766 


.3750 


1 


11 


14 


.3811 


.3750 


0 


12 


E 


.3766 


.3750 


0 


13 


14 


.3 766 


.3750 


0 


14 


15 


.7577 


.7500 


2 


A 


B 


.2520 


.2500 


0 


B 


7 


.2520 


.2500 


0 


C 


D 


.7577 


.7500 


0 


D 


10 


.7577 


.7500 


0 


E 


F 


.3766 


.3750 


0 
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F 


13 


.3766 


.3750 


0 


G 


H 


.2423 


.2500 


0 


E 


15 


.2423 


.2500 


0 



13 



E p , . s . . = 8. 5000 
ij 13 

E (1) = .4048 
E(l)/u = .6538 



B-41 



Module: 1 



Procedure Ho. : 



US 




Nuiiiber of nodes: 


15 




Nuaber of arcs: 


20 




Muaiber of paths: 


7 




'lumber of source stats.: 


1 9 




Average err-cr found: 


0 . 


2217 


Percentage errors found: 


24 


.50 



B-42 



I^odule 1 Procedure 49 





100 


Replications 


100 Repetitions 






i 


p' • . 






1 


2 


1.0000 


1.0000 


2 


2 


3 


.4959 


.5000 


1 


2 


8 


.5041 


.5000 


0 


3 


4 


.2492 


.2500 


2 


3 


8 


.2467 


.2500 


0 


4 


5 


.1260 


.1250 


1 


4 


8 


.1232 


.1250 


0 


5 


6 


.0632 


.0625 


1 


5 


8 


.0628 


.0625 


0 


6 


7 


.0302 


.0313 


1 


6 


10 


.0330 


.0313 


0 


7 


11 


.0164 


.0156 


0 


7 


12 


.0138 


.0156 


2 


8 


A 


.9368 


.9375 


1 


9 


13 


.9368 


.9375 


1 


10 


12 


.0330 


.0313 


3 


11 


12 


.0164 


.0156 


2 


12 


13 


.0632 


.0625 


2 


A 


B 


.9368 


.9375 


0 


B 


9 


.9368 


.9375 


0 










19 


E p. . 


s. . = 5, 


.3751 






E(l) 


= .2560 









E(l)/u = .2829 



B-43 



ttcdule: 1 



Procedure Mo . : 



53 




Nunfcerof nodes: 11 

Mucbercfarcs: 18 

Muttberofpaths: 9 

tlucber of scarce stmts.: 11 

Average errcr found: 0.1876 

Eercentace eircrs round: 35.81 



B-44 



Module 1 



Procedure No . 53 



100 Replications 100 Repetitions 



i 


i 


!ii 




s . . 
iP 


1 


2 


1 - 0000 


1 . 0000 


2 


2 


3 


.4946 


. 5000 


1 


2 


10 


. 5054 


.5000 


0 


3 


4 


.2486 


. 2500 


1 


3 


10 


.2460 


. 2500 


0 


4 


5 


. 1258 


. 1250 


1 


4 


10 


. 1228 


. 1250 


0 


5 


6 


.0632 


.0625 


1 


5 


10 


.0626 


.0625 


0 


6 


7 


.0299 


.0313 


1 


6 


10 


. 0333 


.0313 


0 


7 


8 


.0161 


.0156 


1 


7 


10 


.0138 


.0156 


0 


8 


9 


.0092 


. 0078 


1 


8 


10 


.0069 


. 0078 


0 


9 


10 


. 0040 


. 0039 


0 


9 


11 


.0052 


.0039 


0 


LO 


11 


. 9948 


. 9961 


2 










11 


1 

. p . . s 
ID ID 


i j 


4. 9844 






E (1) = 




. 2374 






E(l)/U 


= 


.4531 







B-45 



Kcdule: 1 



Procedure 




Nu nibe r 


of nodes: 


30 


ti u 03 b e r 


cf arcs: 


35 


tiu [T be r 


of paths: 


12 


fi u E c e r 


cf scarce stats.: 


26 


Av srag 


e error found : 


0.2910 


Per can 


tage errors found: 


23.50 



B-46 



1 

2 

2 

3 

4 

5 

5 

6 

6 

7 

7 

8 

8 

9 

10 

10 

12 

13 

14 

15 

15 

16 

17 



Module 1 Procedure 57 



100 Replications 100 Repetitions 



1 


^ ij 




S^j 


2 


1.0000 


1.0000 


2 


3 


.4972 


.5000 


1 


19 


.5028 


.5000 


1 


A 


.4972 


.5000 


0 


5 


.4972 


.5000 


1 


6 


.2511 


.2500 


1 


9 


.2461 


.2500 


0 


7 


.1228 


.1250 


1 


19 


.1283 


.1250 


0 


8 


.0621 


.0625 


1 


21 


.0607 


.0625 


1 


9 


.0309 


.0313 


0 


19 


.0312 


.0313 


0 


10 


.2770 


.2813 


2 


11 


.1407 


.1407 


0 


12 


.1363 


.1407 


1 


13 


.1644 


.1720 


5 


C 


.1644 


.1720 


0 


15 


.1644 


.1720 


1 


16 


.0842 


.0860 


3 


18 


.0802 


.0860 


0 


E 


.0842 


.0860 


0 


18 


.0842 


.0860 


0 



B-47 



18 


22 


*1644 


.1720 


2 


19 


G 


.6949 


.6875 


0 


20 


22 


.6949 


.6875 


1 


21 


12 


.0281 


.0313 


2 


21 


19 


.0326 


.0313 


0 


E 


F 


.0842 


.0860 


0 


F 


17 


.0842 


.0860 


0 


A 


B 


.4972 


.5000 


0 


B 


4 


.4972 


.5000 


0 


G 


H 


.6949 


.6875 


0 


H 


20 


.6949 


.6875 


0 


C 


D 


.1644 


.1720 


0 


D 


14 


.1644 


.1720 


0 










26 


Z p. . s 
ID 


. . = 7.0874 
ID 








E(l) = 


.3375 








E(l)/u 


= .2726 









B-48 



Module: 1 



Procedure Mo.: 60 




Muahetofncces: 28 

Number cf arcs: 37 

Nunherofpaths: 16 

Number cf scrrce stmts.: 24 

Average errcr found: 0.3336 

Percencace errors found: 29.19 



b-49 




Module 1 Procedure 60 





100 


Replications 


100 Repetitions 




i 


p' . . 




1 


2 


1.0000 


1.0000 


2 


3 


.4987 


.5000 


2 


12 


.5013 


.5000 


3 


4 


.2438 


.2500 


3 


17 


.2549 


.2500 


4 


A 


.2438 


.2500 


5 


6 


.2438 


.2500 


6 


7 


.1225 


.1250 


6 


17 


.1213 


.1250 


7 


8 


.0626 


.0625 


7 


19 


.0599 


.0625 


8 


9 


.0303 


.0313 


8 


12 


.0323 


.0313 


9 


10 


.0172 


.0156 


9 


12 


.0131 


.0156 


10 


11 


.0090 


.0078 


10 


12 


.0082 


.00 78 


11 


12 


.0090 


.0078 


12 


E 


* 


.6095 


13 


14 


* 


.6095 


14 


15 


* 


.3047 


14 


18 


* 


.4063 


15 


C 


* 


.3047 



3 

1 

0 

1 

0 

0 

1 

1 

0 

1 

1 

0 

1 

0 

0 

2 

0 

0 

1 

4 

0 

0 



B-50 



16 


18 


* 


.2188 


1 


17 


18 


.3762 


.3750 


1 


18 


22 


1.0000 


1.0000 


1 


19 


12 


.0295 


.0313 


0 


19 


20 


.0304 


.0313 


2 


20 


C 


.0304 


.0313 


0 


21 


12 


* 


.1641 


1 


C 


D 


•k 


.3282 


0 


D 


16 


* 


.2188 


0 


D 


21 


* 


.1461 


0 


A 


B 


.2438 


.2500 


0 


B 


5 


.2433 


,2500 


0 


E 


F 


★ 


.6095 


0 


F 


13 


* 


.6095 


0 



24 



^ ^ij ^ij = 7.9613 
E(l) = .3791 
ECD/u = .3317 



B-51 



I-lcdule; 1 



Procedure No. : 7 




Number of 


nodes: 


24 


Number cf 


arcs: 


28 


Kunber of 


paths: 


8 


Number of 


source strats.: 


20 


Average error found: 


0.5433 


? e r c e r. t a c 


e errors found; 


5 7 . C 5 



B-52 



i 

1 

2 

2 

3 

4 

5 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

15 

16 

17 

A 

B 

3 



s, 

4 

4 

0 

0 

1 

1 

0 

0 

2 

0 

2 

0 

0 

2 

0 

1 

0 

1 

0 

2 

0 

0 

0 



Module 1 Procedure 75 



100 Replications 100 Repetitions 



i 


p . • 

■■ 




2 


1.0000 


1.0000 


3 


.5020 


.5000 


5 


.4980 


.5000 


A 


.5020 


.5000 


5 


* 


.3750 


6 


* 


.3750 


12 


.5066 


.5000 


C 


* 


.3750 


8 


★ 


. 3750 


A 


* 


.3750 


10 


* 


.3000 


A 


* 


.3000 


12 


.4934 


.5000 


13 


1.0000 


1.0000 


E 


1.0000 


1.0000 


15 


1.0000 


1.0000 


16 


.4937 


.5000 


17 


.5063 


.5000 


17 


.4937 


.5000 


24 


1.0000 


1.0000 


B 


* 


. 7500 


4 


* 


.3750 


9 


* 


.3000 



B- 53 



B 


11 


.4934 


.5000 


0 


E 


F 


1.0000 


1.0000 


0 


F 


14 


1.0000 


1.0000 


0 


C 


D 


* 


.3750 


0 


D 


7 


* 


.3750 


0 



20 



E p. . s. . = 13.6000 

13 13 

E(l) = .6476 
E(l)/u = .6800 



B-54 



Module: 1 



Procedure Ho. : 76 




Hunter of 


nodes: 


15 


liumter cf 


arcs : 


19 


Hunter of 


paths: 


5 


Hunter c f 


sctrce stats.: 


19 


Average e 


rrcr found; 


0.3893 


ifcLCSnCclC 


e errors found; 


43.03 



B-55 







Module 1 Procedure 76 






100 


Replications 100 


Repetitions 




_i 


i 


p' . . 






1 


2 


1.0000 


1.0000 


2 


2 


3 


.4941 


.5000 


2 


2 


10 


.5059 


.5000 


0 


3 


4 


.2431 


.2500 


4 


3 


13 


.2510 


.2500 


0 


4 


A 


.2431 


.2500 


0 


5 


6 


* 


.1250 


3 


6 


A 


* 


.1250 


0 


7 


8 


* 


.1250 


2 


8 


A 


* 


.1250 


0 


9 


10 


.2431 


.2500 


0 


10 


11 


.7490 


.7500 


6 


11 


12 


* 


.3750 


0 


11 


13 


.7490 


.7500 


0 


12 


11 


* 


.3750 


0 


A 


B 


. 7493 


.2500 


0 


B 


5 


* 


.1250 


0 


B 


7 


* 


.1250 


0 


B 


9 


.2431 


.2500 


0 










19 


2 p . . s 
IJ 


ij 


9.1250 






E(l) = 


.4345 






I — 1 

w 


= .4803 







B-56 



Hcdule: 1 



Procedure No . : 11 




IJ u a t e r 


o f 


ncces: 


17 


N u a: fc € r 


cf 


arcs ; 


20 


N u c fc € r 


of 


paths: 


9 


Number 


c f 


scrrce stnts.: 


1 0 


A V e E ag 


S c 


rrci fcunc: 


0.2425 


F e c c G n 


ne c 


e errcrs round: 


50.93 



B-57 




Module 1 Procedure 77 



100 Replications 100 Repetitions 





1 


p • • 
,-^j. 






1 


2 


1.0000 


1.0000 


2 


2 


3 


.49 72 


.5000 


2 


2 


5 


.5028 


.5000 


0 


3 


4 


.2506 


.2500 


0 


3 


5 


.2466 


.2500 


2 


4 


5 


.2506 


.2500 


0 


5 


A 


1.0000 


1.0000 


0 


6 


7 


1.0000 


1.0000 


1 


7 


8 


.4979 


.5000 


1 


7 


9 


.5021 


.5000 


0 


8 


9 


.2513 


.2500 


1 


8 


E 


.2466 


.2500 


0 


9 


C 


. 7534 


.7500 


0 


10 


11 


.7534 


.7500 


1 


C 


D 


.7534 


.7500 


0 


D 


10 


.7534 


.7500 


0 


A 


B 


1.0000 


1.0000 


0 


B 


6 


1.0000 


1.0000 


0 


E 


F 


.2466 


.2500 


0 


F 


11 


.2466 


.2500 


0 










10 


Z p. . 


s. . = 6.0000 
13 








E(l) 


= .2857 









E(l)/u = .6000 



B-58 



acdule: 1 



Procedure No . : 79 




Nua her 


0 L 


nodes: 


25 


Number 


cf 


arcs: 


3 1 


Nua her 


0 £ 


paths: 


3 


N u 3 b 6 r 


of 


soixrce stats . : 


23 


A V erag 


6 error found : 


0.3623 


?arc=nrac= 


‘ errors found; 


33 . 13 



B-59 



Module 1 Procedure 79 





100 Replications 100 


Repetitions 




i 








1 


2 


1.0000 


1.0000 


2 


3 


.4965 


.5000 


2 


E 


.5035 


.5000 


3 


4 


.2458 


.2500 


3 


6 


.2507 


.2500 


4 


A 


.2458 


.2500 


5 


6 


* 


.2500 


6 


7 


* 


.3750 


7 


A 


* 


.3750 


8 


9 


* 


.2500 


9 


A 


* 


.2500 


10 


11 


* 


.2500 


11 


C 


* 


.2500 


12 


13 


* 


.2500 


13 


A 


* 


.2500 


14 


15 


* 


.2500 


15 


A 


* 


.2500 


16 


17 


* 


.2500 


17 


A 


* 


.2500 


18 


25 


.4965 


.5000 


A 


B 


* 


.5000 


B 


5 


* 


.2500 



2 

3 

0 

2 

0 

0 

0 

2 

0 

2 

0 

2 

0 

3 

0 

2 

0 

3 

0 

2 

0 

0 



B-60 



B 


8 


* 


.2500 


0 


B 


10 


* 


.2500 


0 


B 


14 


* 


.2500 


0 


B 


16 


* 


.2500 


0 


B 


18 


.4965 


.5000 


0 


E 


F 


.5035 


.5000 


0 


F 


25 


.5035. 


,5000 


0 


C 


D 


★ 


.2500 


0 


D 


12 


★ 


.2500 


0 










23 


Z p. . s 
i: 


. . = 8.7500 
ID 








E(l) = 


.4167 








E(l)/u 


= .3804 









B-61 



Jicdule; 1 



Procedure llo.: 81 




ilunteroiRCces: 8 

Uurtbercfarcs: 9 

Nunberofpathe: 3 

iluucer cf ecirce stnts.: 7 

Average errcr found: 0.1449 

Perce a cage etrcrs found: 43.>j7 



B-62 



Module 1 Procedure No. 81 





i 






s . . 


1 


2 


(1) 


1.0000 


2 


2 


3 




. 5000 


1 


2 


6 




.5000 


0 


3 


4 




. 2500 


3 


3 


6 




. 2500 


0 


4 


A 




.2500 


0 


A 


B 




. 2500 


0 


B 


5 




. 2500 


0 


5 


6 




. 2500 


1 










7 



) p . , s . . = 

. . ID ID 
ID 


3 .50 


E (1) = 


. 1667 


E (1) /U = 


. 5000 



( 1 ) 






data was not available for this procedure. 



B-63 



II c d u 1 e : 1 



Procedure Ko.: 



87 




l!u [I ber 


of nodes: 


21 


lJu D ber 


of arcs: 


24 


N u E ber 


of paths: 


6 


'i u Ei b e r 


of scarce stats.: 


25 


A verag 


9 error found: 


0.5029 


rce n 


tage errors found; 


4 2.24 



B-64 



1 

2 

3 

4 

4 

5 

5 

6 

6 

7 

8 

9 

10 

11 

11 

12 

13 

14 

E 

F 

A 

B 



Module 1 Procedure 87 



lOQ Replications 100 Repetitions 



i 


p • • 






2 


1.0000 


1.0000 


2 


A 


1.0000 


1.0000 


0 


4 


1.0000 


1.0000 


4 


5 


.4915 


.5000 


1 


10 


.5085 


.5000 


0 


6 


.2438 


.2500 


2 


12 


.2477 


.2500 


0 


7 


.2572 


.2500 


0 


8 


.2444 


.2500 


3 


8 


.2572 


.2500 


0 


C 


.5016 


.5000 


0 


21 


.5016 


.5000 


1 


11 


.5085 


.5000 


2 


6 


.2578 


.2510 


4 


21 


.2507 


.2500 


0 


13 


.2477 


.2500 


4 


E 


.2477 


.2500 


0 


21 


.2477 


.2500 


2 


F 


.2477 


.2500 


0 


14 


.2477 


.2500 


0 


B 


1.0000 


1.0000 


0 


3 


1.0000 


1.0000 


0 



B-65 



.5000 0 

.5000 0 

25 

S p. . s, . = 11.7500 
ID ID 

E(l) = .5595 
E(l)/u = .4700 



C D .5016 

D 9 .5016 



B-66 



Ilcdule: 1 



Procedure No . : 91 




Nuobec of 


nodes: 


22 


Nuaber of 


arcs: 


30 


Nuibsr of 


paths: 


9 


Number c f 


soi^rce stobs.: 


14 


Average erroi found: 


0- 1438 


Perc e nt a g 


e errors found: 


2 1.57 



B-67 




1 

2 

2 

3 

3 

4 

4 

5 

5 

6 

6 

7 

8 

9 

9 

10 

10 

11 

12 

13 

13 

14 

15 



2 

1 

0 

1 

0 

1 

0 

1 

0 

1 

0 

0 

1 

1 

0 

1 

0 

0 

1 

0 

0 

1 

0 



Module 1 Procedure 91 



100 Replications 




2 1.0000 



3 


.4957 


22 


.5043 


4 


• .2494 


22 


.2463 


5 


.1254 


17 


. 12 40 


6 


.0628 


14 


. 0626 


7 


.0301 


22 


.0327 


A 


.0301 


9 


* 


10 


* 


14 


.0085 


11 


* 


22 


.0054 


20 


* 


13 


.0162 


14 


.0082 


22 


.0080 


15 


.0793 


18 


.0793 



100 Repetitions 




1.0000 
.5000 
.5000 
.2500 
.2500 
.1250 
.1250 
.0625 
.0625 
.0313 
.0313 
.0313 
.0157 
.0078 
.0078 
.00 39 
.0039 
.0039 
.0153 
.0078 
.0078 
.0860 
.0860 



B-68 



16 


17 


.0793 


.0860 


0 


17 


22 


.2033 


.2110 


2 


C 


D 


.0793 


.0860 


0 


D 


16 


.0793 


.0860 


0 


A 


B 


.0348 


.0313 


0 


B 


8 


.0186 


.0157 


0 


B 


12 


.0162 


.0157 


0 










14 


Z p. . s 


. . = 3.5195 








E(l) = 


.1676 








E(l)/u 


= .2514 









B-69 



Hcdule: 1 



Procedure N'o. : 



92 




iJusfcercfccces: 5 

Nucfcer of arcs: 6 

i^urtteroi paths: 3 

lluahsr cf sc tree stats.: 9 

Average errer feund: 0.1837 

r s r cer. ta g s s r c ers iou ad : 4 2.6b 



B-70 



Module 1 



Procedure No. 92 



500 Replications 



500 Repetitions 



i i 


"li 






1 2 


1 .0000 


1 . 0000 


2 


2 3 


.4995 


. 5000 


1 


2 5 


. 5005 


.5000 


0 


3 4 


. 2499 


. 2500 


0 


3 5 


. 2497 


. 2500 


6 


4 5 


. 2499 


. 2500 


0 








9 


1 

P . . s . . = 

13 13 13 


4 .0 






E(l) = 


. 1905 






E (1) /U = 


.4444 







B-71 



H c d u 1 e : 1 



Procedure No.; 93 




N u a £ € r 


o f 


nodes: 


25 


liu a ter 


c f 


arcs : 


34 


N u a: t e r 


o f 


f a t hs : 


12 


N u I b 6 r 


cf 


scrrce stmts.: 


34 


Ave rag 


e errcr found; 


0.3972 


E e r c e n 


ta c 


e errors found: 


24.53 



b-72 



1 

2 

2 

3 

3 

4 

4 

5 

5 

6 

6 

7 

7 

8 

9 

10 

10 

11 

12 

13 

14 

15 



Module 1 Procedure 93 



100 


Replications 100 


Repetitions 




1 


ill 






2 


1.0000 


1.0000 


2 


3 


.4996 


.5000 


1 


19 


.5004 


.5000 


0 


4 


.2537 


.2500 


1 


.20 


.2459 


.2500 


0 


5 


.1301 


.1250 


2 


20 


.1236 


.1250 


0 


6 


.0659 


.0625 


8 


23 


.0642 


.0625 


0 


7 


.0339 


.0313 


2 


9 


.0320 


.0313 


0 


8 


★ 


.0157 


0 


12 


.0339 


.0313 


1 


7 


★ 


.0157 


0 


10 


.0320 


.0313 


2 


11 


★ 


,0157 


0 


12 


.0320 


.0313 


0 


10 


* 


.0157 


0 


13 


.0659 


.0625 


5 


A 


.0659 


.0625 


0 


15 


★ 


.3125 


2 


16 


★ 


. 1563 


0 



B-7 3 



15 


17 


* 


.3125 


2 


16 


15 


* 


.1563 


0 


17 


A 


* 


.3125 


0 


18 


23 


.4629 


.4688 


1 


19 


21 


.5004 


.5000 


1 


20 


21 


.3695 


.3750 


3 


21 


A 


.8699 


.8750 


0 


22 


23 


.4729 


.4688 


1 


A 


B 


* 


.9375 


0 


B 


14 


* 


.3125 


0 


B 


18 


.4629 


.4688 


0 


B 


22 


.4729 


.4688 


0 



34 



Z p. . s. . = 7.7814 
E(l) = .3705 
E(l)/u = .2289 



B-74 



Mcdule: 1 



Procedure Ho. : 



95 




Kuater of 


ncces: 


18 


Hu a ter of 


arcs: 


27 


Hunter of 


paths: 


10 


H u 0 1 6 n c f 


source stmts.: 


19 


Average et 


r c r found: 


0. 1822 


Percentace 


srccrs found: 


20.14 



B-75 



Module 1 Procedure 95 



100 Replications 100 Repetitions 





i 








1 


2 


1.0000 


1.0000 


2 


2 


3 


.4964 


.5000 


2 


2 


16 


.50 36 


.5000 


0 


3 


4 


.2470 


.2500 


1 


3 


12 


.2494 


.2500 


1 


4 


5 


.1229 


.1250 


1 


4 


16 


.1241 


.1250 


0 


5 


6 


.0633 


.0625 


1 


5 


13 


.0596 


.0625 


0 


6 


7 


.0315 


.0313 


1 


6 


16 


.0318 


.0313 


0 


7 


8 


.0166 


.0156 


1 


7 


16 


.0149 


.0156 


0 


8 


9 


.0087 


.0078 


1 


8 


16 


.00 79 


.0078 


0 


9 


10 


.0042 


.0039 


3 


9 


13 


.0045 


.0039 


0 


10 


A 


.0042 


.0039 


0 


11 


16 


.0978 


.0977 


1 


12 


13 


.1255 


.1250 


0 


12 


16 


.1239 


.1250 


3 


13 


14 


.1896 


.1914 


0 



B-76 



14 


A 


.1896 


.1914 


1 


15 


16 


.0960 


.0977 


0 


A 


B 


.1938 


.1953 


0 


B 


11 


.0978 


.0977 


0 


B 


15 


.0960 


.0977 


0 



19 



y p. . s. . = 4.4180 
^ x: ij 

E(l) = .2104 
E(l)/u = .2325 



B-77 



Module : 



Procedure No. 15 







li usher cf 


DQcJes: 


1 1 


N u c h s r c f 


a res : 


13 


N usher cf 


paths: 


5 


Nuaher cf 


serree stats.: 


10 


Average errct found: 


0.0836 


Perce ncac 


e exrcrs found; 


42.64 



B-78 



MODULE 2 



Procedure 15 





100 


Replications 


100 Repetitions 








pV . 




S . . 


i 


i 






13 


1 


2 


1.0000 


1.0000 


0 


2 


3 


.4944 


.5000 


3 


2 


9 


.5056 


.5000 


0 


3 


4 


.4992 


.2500 


0 


3 


5 


.4994 


.5000 


2 


4 


3 


.4992 


.2500 


0 


5 


6 


•* 


.5000 


3 


6 


A 


* 


.5000 


0 


7 


8 


★ 


.5000 


1 


/ 










8 


5 


★ 


.2500 


1 


8 


9 


.4944 


.5000 


0 


A 


B 


* 


.5000 


0 


B 


7 


* 


.5000 


0 



10 



y p. ,s . . = 4. 7500 
^ 13 13 

E(l) = .0931 

E(l)/u = .4750 



B-79 



Module: 2 



Procedure No.: 23 




Nucter of 


ncc€s: 


10 


Nunher of 


arcs : 


1 1 


Nunfcer of 


paths: 


3 


Nuifcer cf 


scr rce stats. : 


12 


Average error found: 


0-1592 


Peicsntac 


e errors found; 


5 7.65 



B-80 



MODULE 2 Procedure 23 



100 Replications 100 Repetitions 



1 

2 

2 

3 

3 

4 

5 

6 
7 
A 
B 



i 


p • • 






2 


1.000 


1.000 


3 


3 


.4947 


.5000 


1 


4 


.5053 


.5000 


0 


4 


.2497 


.2500 


1 


6 


.2450 


.2500 


1 


5 


.7550 


.7500 


3 


8 


.7550 


.7500 


2 


A 


.2450 


.2500 


0 


8 


.2450 


.2500 


1 


B 


.2450 


.2500 


0 


7 


.2450 


.2500 


0 



12 



y p. . s . . = 8. 0000 
ID ID 

E(l) = .1569 

E(l)/u = .6667 



B-81 



Bed uls : 






Procedure tio . ; 4 1 




Nun her of 


ncc€s: 


10 


Nuffher cf 


arcs : 


1 4 


Nunher of 


paths: 


12 


Nunher cf 


scLice stcats,: 


13 


Average error found: 


0. 1554 


rGHC^nilciG 


e errors found: 


5C .56 



B-82 



Module 2 



Procedure 41 





100 


Replications 


100 Repetitions 




i 


i 


p' • . 
ID 


^iD 


s . . 


1 


2 


1.0000 


1.0000 


3 


2 


3 


.4955 


.5000 


0 


2 


6 


.5045 


" .5000 


0 


3 


4 


★ 


.5000 


2 


4 


5 


★ 


.2500 


3 


4 


6 


.3303 


.3333 


0 


5 


3 


★ 


.1250 


0 


5 


6 


.1652 


.1667 


0 


6 


7 


.4993 


.5000 


1 


6 


10 


.5007 


.5000 


0 


7 


8 


.4993 


.5000 


2 


8 


9 


.2474 


.2500 


0 


8 


10 


.2519 


.2500 


2 


9 


10 


.2474 


.2500 


0 










13 


y p. ,s, . =6.75 
^ 13 13 










E(l) = .1324 










E(l)/u = .5192 











B-83 



Modu 1 e : 2 



Procedure No . 46 




Number of nodes: 23 

Number of arcs: 32 

Number of paths: 18 

Number of source stmts.: 32 

Average error found: 0.1657 

Percentage errors found: 26.41 



B-84 




1 

2 

5 

6 

7 

7 

8 

9 

9 

10 

10 

11 

11 

12 

12 

13 

14 

14 

15 

15 

16 

16 

17 

la 



Module 2 



Procedure 46 



100 


Replications 


100 Repetitions 






p* . . 


p. • 


s , . 


i 








2 


1.0000 


1.0000 


3 


6 


1.0000 


1.0000 


0 


23 


.3908 


.3906 


2 


7 


1.0000 


1.0000 


2 


8 


.4955 


.5000 


2 


23 


.5045 


.5000 


0 


9 


.4955 


.5000 


2 


10 


.2485 


.2500 


1 


14 


.2470 


.2500 


1 


5 


.1220 


.1250 


0 


11 


.1265 


.1250 


2 


5 


.0642 


.0625 


0 


12 


.0623 


.0625 


4 


5 


.0333 


.0313 


0 


13 


.0290 


.0313 


2 


5 


.0290 


.0313 


0 


15 


.1228 


.1250 


1 


21 


.1242 


.1250 


1 


16 


.0607 


.0625 


1 


21 


.0621 


.0625 


0 


5 


.0330 


.0313 


0 


17 


.0277 


.0313 


1 


24 


.0277 


.0313 


0 


19 


.1051 


.1094 


0 



B-85 





19 


20 


.2140 


.2188 


3 




20 


5 


.1093 


.1094 


0 




20 


23 


.1047 


.1094 


3 




21 


24 


.1863 


.1875 






22 


19 


.1089 


.1094 


1 




24 


25 


.2140 


.2188 


0 




25 


18 


.1051 


.1094 


0 




25 


22 


.1089 


.1094 


0 












32 


I p. .s . . 


= 10.2815 










E(l) 


= .2016 










E(l)/u 


= .3213 











B-86 



Mod ule: 



Procedure No.; 47 




Nuaber of nodes: 


24 




iiuober of arcs: 


35 




Nucber of paths: 


36 




Husber cf scarce stmts.; 


33 




Average error found: 


0. 


1 163 


Percentage errors found: 


17 


.97 



B-87 





1 

2 

2 

3 

3 

4 

4 

5 

5 

6 

6 

7 

7 

8 

8 

9 

9 

10 

10 

11 

11 

12 

13 



Module 2 



Procedure 47 



100 


Replications 


100 Repetitions 




i 


p' ■ • 






2 


1.0000 


1.0000 


3 


3 


.4915 


.5000 


2 


22 


-5085 


.5000 


0 


4 


.2443 


. 2500 


2 


5 


• 2472 


.2500 


0 


5 


.1231 


. 1250 


2 


22 


.1212 


.1250 


0 


6 


.1874 


.1875 


2 


7 


.1829 


.1875 


0 


7 


.0928 


.0938 


1 


22 


.0946 


.0938 


0 


8 


.1403 


.1406 


1 


21 


.1354 


.1406 


0 


9 


.0719 


.0703 


1 


21 


.0684 


.0703 


0 


10 


.0352 


.0352 


1 


21 


.0367 


.0352 


0 


11 


.0184 


.0176 


1 


18 


.0168 


.0176 


0 


12 


.0089 


.0088 


0 


22 


.0095 


.0088 


1 


13 


.0089 


.0088 


3 


A 


.0089 


.0088 


0 



B-88 





14 


15 


.0127 


.0132 


1 




15 


16 


.0257 


.0268 


2 




16 


17 


* 


.0132 


0 




16 


22 


.0257 


.0264 


1 




17 


16 


* 


.0132 


0 




18 


19 


.0168 


.0176 


3 




19 


A 


.0168 


.0176 


0 




20 


15 


.0130 


.0132 


2 




21 


22 


.2405 


.2461 


4 




A 


B 


.0257 


.0264 


0 




B 


14 


.0127 


.0132 


0 




B 


20 


.0130 


.0132 


0 












33 


1 P. .3. . 
^ ID ID 


= 6.7983 










Ed) 


= .1333 










E(l)/u 


= .2060 











B-89 



Ucdule: 2 



Procedure Mo.: 48 




M u CL t e r 


0 f 


n c c 6 s : 


16 


M u E b e r 


of 


arcs : 


21 


Nu c ber 


0 f 


paths: 


14 


M u E fc e c 


cf 


sc tree stu.ts.: 


17 


Av erag 


e errcr found: 


0. 1530 


? € E c e n 


C Q C 


e e ircrs found: 


4 7 . 4 C 



B-90 



Module 2 



Procedure 48 





100 


Replications 


100 Repetitions 








P‘ . . 


P* • 


s . . 


i 


i 




liJL 




1 


2 


1.0000 


1.0000 


3 


2 


3 


.4934 


.5000 


2 


2 


14 


.5066 


.5000 


0 


3 


4 


.2444 


.2500 


2 


3 


5 


.2490 


.2500 


0 


4 


5 


.1229 


.1250 


2 


4 


14 


.1215 


.1250 


0 


5 


6 


.1869 


.1875 


0 


5 


7 


.1850 


.1875 


1 


6 


7 


.1869 


.1875 


0 


7 


8 


.3719 


.3750 


2 


8 


A 


.3719 


.3750 


0 


9 


10 


.3719 


.3750 


1 


10 


11 


.1863 


.1875 


1 


10 


13 


.1856 


.1875 


0 


11 


12 


.0979 


.0938 


0 


11 


13 


.0884 


.0938 


1 


12 


13 


.0979 


.0938 


0 


13 


14 


.3719 


.3750 


2 


A 


B 


.3719 


.3750 


0 


B 

1 - 7.9063 

E(l) = .1550 

E(l)/U = .4651 


9 


.3719 


.3750 


0 

17 



B-91 



Module: 2 Procedure t.o.: 79 




M u c fc e r 


0 f 


DC des: 


13 


i.’ u c t e r 


cf 


ar cs : 


14 


N u c t e r 


c f 


paths; 


5 


i.' u d: h 6 r 


cf 


sctrce stats. : 


1 1 


A V e r a g 


9 err Cl f curd ; 


0. 1379 



Psrcer.tacs eircrs found: 53.5^ 



B-92 



Module 2 



Procedure 79 



100 Replications 100 Repetitions 





i 


p • • 




s . . 


1 


2 


1.0000 


1.0000 


1 


2 


3 


* 


1.0000 


2 


3 


4 


★ 


.5000 


2 


3 


9 


1.0000 


1.0000 


0 


4 


2 


•k 


.3333 


1 


4 


5 


★ 


.3333 


0 


5 


A 


★ 


.3333 


4 


6 


7 


★ 


.3333 


0 


7 


C 


* 


.3333 


1 


8 


2 


* 


.3333 


0 


A 


B 


* 


.3333 


0 


B 


6 


★ 


.3333 


0 


C 


D 


* 


.3333 


0 


D 


8 


* 


.3333 


0 










11 


y p. .s . . = 6.0000 
13 13 










E(l) = .1176 










E(l)/u = .5455 











B-93 



Module : 



Procedure No 



86 




Nunfcer of 


ncG€s: 


30 


Nucter of 


arcs: 


34 


No aber of 


paths: 


6 


l) u c 1 6 r c f 


s c i: IC6 stats-: 


33 


Average er 


rci found: 


0.2042 


Percentage 


silOits jZOuHI: 


3 1.56 



B-94 



1 

2 

2 

3 

4 

5 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

16 

17 

17 

18 

19 

A 

B 



s 

2 

2 

1 

0 

1 

0 

1 

0 

3 

0 

7 

0 

2 

4 

0 

0 

2 

1 

0 

0 

5 



Module 2 



Procedure 86 



i 


p* . , 




2 


1.0000 


1.0000 


3 


.4962 


.5000 


6 


.5038 


.5000 


A 


.4962 


.5000 


5 


.4996 


.5000 


12 


.2524 


.2500 


20 


.2472 


.2500 


C 


.5038 


.5000 


8 


.5038 


.5000 


E 


.5038 


.5000 


10 


.5038 


.5000 


A 


.5038 


.5000 


20 


.5004 


.5000 


13 


.2524 


.2500 


G 


.2524 


.2500 


15 


.2524 


.2500 


16 


★ 


.2500 


17 


* 


.1250 


20 


.2524 


.2500 


15 


★ 


.0833 


18 


★ 


.0833 


I 


★ 


.0833 


15 


* 


.0833 


B 


1.0000 


1.0000 


4 


.4996 


.5000 



B-95 



B 


11 


.5004 


.5000 


0 


C 


D 


.5038 


.5000 


0 


D 


7 


.5038 


.5000 


0 


E 


F 


.5038 


.5000 


0 


F 


9 


.5038 


.5000 


0 


G 


H 


.2524 


.2500 


0 


H 


14 


.2524 


.2500 


0 


I 


J 


ic 


.0833 


0 


J 


19 


★ 


.0833 


0 



33 



y p. .s, . = 12.4582 
^ ID ID 

E(l) = .2443 

E(l)/u = .3775 



B-96 



Kcd ule: 



Procedure No.: 90 




Nun her of 


ncces: 


13 


Nucher cf 


arcs : 


18 


Munter of 


paths: 


8 


Nunber cf 


source stats. : 


23 


Average errci found; 


0-0958 




e e rrcrs found ; 


2 1 . 2 



B-97 



Module 2 



Procedure 90 







100 


Replications 


100 Repetitions 






i 


i 


p' . . 

13 




s , . 

13 




1 


2 


1.0000 


1.0000 


2 




2 


3 


.4925 


.5000 


1 




2 


11 


.5075 


.5000 


0 




3 


4 


.2427 


.2500 


1 




3 


5 


.2498 


.2500 


0 




4 


5 


.1249 


.1250 


1 




4 


11 


.1178 


.1250 


2 




5 


A 


.3747 


.3750 


0 




6 


7 


* 


.2500 


3 




7 


8 


* 


.1875 


3 




7 


11 


.3747 


.3750 


0 




8 


7 


* 


.0938 


8 




8 


9 




.1875 


1 




9 


A 


* 


.1875 


0 




10 


7 


* 


.2500 


1 




A 


B 


* 


.3750 


0 




B 


6 


* 


.2500 


0 




B 


10 


* 


.2500 


0 












23 


I P- S . 
^ iD 


= 5.625 










E(l) 


= .11029 










E(l)/u 


= .2446 











B-98 



Mo du 1 e : 2 



Procedure No.: 




Number of arcs: 30 

Number of paths: 10 

Number of source stmts.: 23 

Average error found: 0.1513 

Percentage errors found: 33.55 



B-99 














i 

1 

2 

2 

3 

3 

4 

4 

5 

5 

6 

7 

8 

9 

10 

10 

11 

12 

12 

13 

14 

15 

16 

17 

18 



MODULE 2 



Procedure 99 



100 


Replications 


100 Repetitions 






P' . ■ 


P. . 


s . . 


i 






ID 


2 


1.0000 


1.0000 


2 


3 


.4981 


.5000 


1 


11 


.5019 


.5000 


0 


4 


.2506 


.2500 


1 


18 


.2475 


.2500 


0 


5 


.1307 


.1250 


2 


6 


.1199 


,1250 


0 


6 


.0643 


.0625 


2 


18 


.0664 


.0625 


0 


7 


.6861 


.6875 


0 


8 


* 


.6875 


3 


A 


* 


.6875 


0 


10 


* 


.6875 


1 


7 


★ 


.3438 


0 


18 


.6861 


.6875 


0 


12 


.5019 


.5000 


2 


13 


.2483 


.2500 


3 


17 


.2536 


,2500 


0 


E 


.2483 


.2500 


0 


15 


.2483 


.2500 


2 


G 


.2483 


.2500 


0 


17 


.2483 


.2500 


0 


6 


.5019 


.5000 


3 


19 


1.0000 


1.0000 


1 



B-lOO 



A 


B 




★ 


.6875 


0 


B 


9 




* 


.6875 


0 


E 


F 




.2483 


.2500 


0 


F 


14 




.2483 


.2500 


0 


G 


H 




.2483 


.2500 


0 


H 


16 




.2483 


.2500 


0 












23 


E p. .S . . 
^ ID ID 


= 


10.625 








E(l) 


= 


.20833 








E(l)/U 


= 


.4620 









B-101 



tlcdule: 2 



Procedure No.: 122 




Nu 3 te r 


c f 


nodes: 


18 


N u c t e r 


cf 


arcs: 


21 


Nuater 


o f 


paths: 


6 


N u c t e r 


cf 


settee stnts.: 


20 


Average 


errct found: 


0. 1685 


Piir cen t 


a c 


c errors found; 


4 2 . S S 



B-102 



Module 2 



Procedure 122 





100 


Replications 


100 Repetitions 




i 


i 


p’ • • 






1 


2 


1.0000 


1.0000 


3 


2 


3 


.4945 


,5000 


1 


2 


12 


.5055 


.5000 


1 


3 


4 


.2480 


.2500 


1 


3 


5 


.2465 


.2500 


1 


4 


5 


.1232 


.1250 


2 


4 


12 


.1248 


.1250 


1 


5 


6 


★ 


.1875 


0 


5 


8 


.3697 


.3750 


4 


6 


A 


* 


.1875 


0 


7 


5 


★ 


.1875 


0 


8 


C 


.3697 


.3750 


0 


9 


10 


.3697 


.3750 


5 


10 


E 


.3697 


.3750 


0 


11 


12 


.3697 


.3750 


1 


A 


B 


★ 


.1875 


0 


B 


7 


★ 


.1875 


0 


C 


D 


.3697 


.3750 


0 


D 


9 


.3697 


.3750 


0 


E 


F 


.3697 


.3750 


0 


F 


11 


.3697 


.3750 


0 



20 



y p. .s. . = 8.625 
^ 13 13 

E(l) = .1691 



E(l)/u 



.4313 



B-103 



Hcc ule ; 



2 



Procedure Mo. : 



157 




N u n h e r 


o f 


ncc es: 


13 


cl u E t € r 


cf 


arcs; 


17 


M u tr t e r 


o i 


fa ths: 


1 1 


M u E t e c 


c f 


scrrce stnts.: 


23 


A V e E a g 


e exccE found: 


0.1178 


2 e r c e n 


t ac 


6 errors found: 


2 6.12 



B-104 



MODULE 2 



Procedure 137 





100 


Replications 


100 Repetitions 




i 


i 


p' • ■ 


!ii 


s . . 


1 


2 


1.0000 


1.0000 


2 


2 


3 


.4994 


.5000 


1 


2 


9 


.5006 


.5000 


1 


3 


4 


.2482 


.2500 


0 


3 


5 


.2512 


.2500 


1 


4 


5 


.2482 


.2500 


0 


5 


6 


.4994 


.5000 


2 


6 


7 


.2503 


.2500 


2 


6 


8 


.2491 


.2500 


0 


7 


8 


.1269 


.1250 


1 


7 


9 


.1234 


.1250 


0 


8 


9 


.1863 


.1875 


0 


8 


11 


.1897 


.1875 


12 


9 


12 


.8103 


.8125 


0 


10 
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